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EXECUTIVE  SUMMARY 


The  purpose  of  the  Operations  in  Urban  Environment  Collaboration  Framework  (OUECF)  is  to 
explore  and  demonstrate  technologies  and  capabilities  for  urban  operations  in  the  context  of 
operationally  relevant  scenarios.  The  technology  collaboration  between  U.S.  Pacific  Command 
(USPACOM)  and  Singapore  Ministry  of  Defence  (MINDEF)/Singapore  Armed  Forces  (SAF)  was 
inaugurated  in  2002  as  a  Transformation  Working  Group  (TWG)  under  the  ambit  of  the  Annual  Staff 
Talk.  The  Operations  in  Urban  Environment  Collaboration  (OUEC)  was  established  in  2013  under 
the  sphere  of  the  Capability  Development  Working  Group  (CDWG).  Previous  OUE  events  focused 
on  testing  and  demonstrating  smart  devices  as  prototypes  for  tactical  Command,  Control, 
Communications,  and  Computers  (C4),  improving  situational  awareness  for  warfighters  in  HADR 
environments,  and  persistent  surveillance  in  both  scripted  scenarios  and  live  exercises. 

OUE  14.2  was  conducted  in  Piedra  del  Lumbre  (PDL)  Combat  Town,  Camp  Pendleton,  CA  from  5- 
19  December  2014  in  conjunction  with  Exercise  Valiant  Mark  14.2.  The  event  consisted  of  an 
integrated  demonstration  of  the  Persistent  Ground  Surveillance  System  (PGSS),  4G  LTE 
communications,  video  analytics,  and  the  DAVION  and  PGSS  Mobile  smart  phone  applications. 

The  event  was  divided  into  3  main  focus  areas;  the  integration  of  the  Singaporean  and  U.S. 
technologies,  data  and  feedback  collection  on  the  DAVION  applications,  and  data  and  feedback 
collection  on  the  PGSS  Mobile  Client.  Users  from  the  Singaporean  Guards  and  Alpha  Company,  1st 
Battalion,  5th  Marine  Regiment  participated  in  scripted  vignettes  to  gain  exposure  to  the  application 
suites  and  provide  feedback  on  their  utility.  Users  were  also  equipped  with  the  ISR  integrated 
smartphones  and  software  during  the  mechanized  assault  conducted  on  14  December  2014. 

On  19  December  2014  a  Distinguished  Visitors  (DV)  Day  was  held  to  demonstrate  the  successful 
integration  and  utility  of  the  OUE  14.2  technologies.  A  combined  team  of  Singaporean  Guards  and 
U.S.  Marines,  using  the  DAVION  and  PGSS  handsets,  successfully  conducted  a  live  demonstration 
of  some  of  the  features  provided  by  the  applications,  while  utilizing  the  PGSS  network  and  Video 
Analytics  software  to  support  the  scenario.  The  event  was  attended  by  over  60  guests  including; 
Major  General  Lawrence  D.  Nicholson,  Commanding  General,  1st  Marine  Division,  COL  Lim  Siong 
Tiong,  Head  Concept  Generation  Group,  Future  Systems  and  Technology  Directorate,  COL  Tan 
Cheng  Kwee,  Commander  7  Singapore  Infantry  Brigade  (7  SIB),  Sergeant  Major  David  L.  Jobe, 
Sergeant  Major,  1st  Marine  Division,  and  other  military  and  civilian  visitors  of  note. 

The  integration  Subject  Matter  Experts  (SME),  Singaporean  Guards,  and  U.S.  Marine  users  all 
provided  feedback  on  the  technologies  that  were  inserted  in  OUE  14.2.  Overall  the  SMEs  viewed  the 
integration  of  the  technologies  as  a  success,  and  through  their  efforts  exhibited  the  value  of 
collaboration  by  working  together  across  technologies  and  organizations  to  address  any  issues  as  they 
arose.  The  Singaporean  Guards  and  U.S.  Marines  provided  the  technologists  with  valuable  feedback 
on  the  functionality,  possible  improvements,  and  streamlining  of  the  capabilities  of  the  DAVION 
applications  and  PGSS  Mobile  Client.  Overall  the  users  were  very  happy  with  the  technologies  and 
felt  each  would  be  a  valuable  asset  for  decision  making  and  situational  awareness  in  almost  any 
environment. 

Overall,  OUE  14.2  was  a  successful  event  for  technology  insertion,  integration,  and  partner  nation 
S&T  collaboration  efforts.  The  data  collected  from  this  event  will  help  shape  continued  technology 
development  for  our  warfighters  and  future  PACOM  S&T  engagement  efforts. 
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Purpose 


INTRODUCTION 


The  purpose  of  the  Operations  in  Urban  Environment  Collaboration  Framework  (OUECF)  is  to 
explore  and  demonstrate  technologies  and  capabilities  for  urban  operations  in  the  context  of 
operationally  relevant  scenarios.  These  scenarios  articulate  common  challenges  faced  by  troops 
operating  within  the  urban  environment.  The  Technology  Experimentation  Center  (TEC)  and 
Future  Systems  and  Technology  Directorate  (FSTD)  worked  in  concert  in  capability 
development  by  harnessing  respective  expertise  and  tapping  on  respective  technology 
ecosystems  to  overcome  or  mitigate  the  identified  operational  challenges  in  the  defined 
scenarios. 

Background 

The  technology  collaboration  between  US  Pacific  Command  (USPACOM)  and  Singapore 
Ministry  of  Defence  (MINDEF)/Singapore  Armed  Forces  (SAF)  was  inaugurated  in  2002  as  a 
Transformation  Working  Group  (TWG)  under  the  ambit  of  the  Annual  Staff  Talk.  The 
Capability  Development  Working  Group  (CDWG)  replaced  TWG  in  2007  with  the  intention  to 
widen  the  scope  of  collaboration  to  include  technology  development  and  capability 
demonstration  based  on  operational  challenges  common  to  both  USPACOM  and  the 
MINDEF/SAF. 

The  Operations  in  Urban  Environment  Collaboration  (OUEC)  is  established  between  U.S. 
PACOM,  and  Singapore  MINDEF  as  a  collaboration  track  under  the  ambit  of  CDWG.  OUEC 
will  adopt  a  multi-year  spiral  approach  towards  capability  development  for  operations  in  urban 
environment.  Technologies  and/or  prototypes  will  be  demonstrated  in  field  environment  to  seek 
operational  feedback  that  contributes  to  enhancing  development  and  facilitating  the  eventual 
transition  of  these  technologies. 

To  kick-start  the  collaboration,  a  technology  demonstration  (TD)  that  successfully  demonstrated 
the  use  of  smart  devices  (ipads  and  iphones)  as  prototype  tactical  C4  systems  was  conducted 
during  the  Area  Security  Operations  phase  of  Exercise  Valiant  Mark  (VM)  1 1 .  Going  ahead,  the 
TEC  and  FSTD,  built  on  this  initial  success  and  continued  to  leverage  various  plausible 
platforms  to  enhance  and  further  develop  capabilities.  The  2012  Operations  in  Urban 
Environments  (OUE)  TD  focused  on  improving  situational  awareness  for  warfighters  in 
Humanitarian  Assistance  Disaster  Relief  (HADR)  environments  and  identifying  opportunities  to 
improve  interoperability  between  the  U.S.  and  Singaporean  militaries.  From  January  13-25  an 
OUE  TD  was  conducted  during  exercise  VM14.1  venue  in  western  Singapore.  U.S.  and 
Singapore  technologies  together  demonstrated  a  persistent  surveillance  capability  in  both 
scripted  scenarios  and  during  observation  of  VM  14.1  forces.  As  the  final  phase,  OUE  14.2, 
USPACOM  and  MINDEF  conducted  an  integrated  demonstration  of  the  Persistent  Ground 
Surveillance  System  (PGSS),  4G  LTE  communications,  video  analytics,  and  smart  phone 
technologies. 

Valiant  Mark  14.2  Objectives 

Valiant  Mark  is  an  annual  bilateral  training  exercise  conducted  between  Singapore  Armed  Forces 
(SAF)  and  I  Marine  Expeditionary  Force  (MEF).  Historically  VM  has  been  conducted  in  a 
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variety  of  methods  ranging  from  a  Battalion  (BN)  level  Command  Post  Exercise  (CPX) 
conducted  in  Singapore  to  a  Commanding  Officer  (CO)  level  Field  Training  Exercise  (FTX) 
conducted  in  the  US  and  Singapore.  Beginning  in  2014,  the  VM  series  of  exercises  will  be 
focused  solely  on  full  spectrum,  combined  arms  training.  The  exercises  will  run  on  a  two  year 
cycle.  The  first  event  of  2014  (VM  14.1)  was  conducted  in  Singapore  with  company  level 
training  focused  on  individual  and  unit  live  fire,  helibome  operations  and  Military  Operations  in 
Urban  Terrain  (MOUT).  The  second  year  will  was  conducted  in  the  US  with  BN-Regiment 
(REGT)  level,  high  end,  combined  arms  live  fire  training.  VM  14.2  was  scheduled  to  take  place 
in  December  2014  in  conjunction  with  exercise  Steel  Knight,  a  division  level  combined  arms 
exercise  in  29  Palms  and  Camp  Pendleton. 

VM  14.2  Exercise  Objectives: 

•  Develop  and  enhance  full  spectrum  combined  arms  core  competencies 

•  Conduct  company  level  live  fire  and  maneuver 

•  Conduct  MOUT 

•  Conduct  mechanized  Ops 

•  Conduct  sniper  training 

•  Conduct  engineer  training 

•  Conduct  120mm  mortar  training 

•  Conduct  SAF  BDE/REGT  staff  augmentation/planning 

•  Conduct  SAF  aviation  observation  (in  preparation  for  VM  2016) 

•  Develop  and  enhance  U.S.  Marine  Corp  (USMC)/SAF  command  and  control  in  a 
combined  environment 

•  Enhance  USMC-SAF  interoperability 

•  Improve  USMC-SAF  mil-to-mil  relationship 

General  OUECF  Objectives 

This  section  outlines  the  overall  OUECF  objectives.  These  objectives  have  three  key  focus 
areas:  Surveillance,  C4,  and  Response. 

Surveillance 

•  Unmanned  Ariel  Vehicle  (UAV)  to  capture  imaging  and  video  streams  by  flying  between 
buildings,  and  perching-and-staring  at  critical  junctions  or  areas  for  persistent 
surveillance 

•  Smartphones  to  capture  street  level  images  and  video  carried  by  the  operators  to  realize 
the  concept  of  'soldier  sensor',  supported  by  Intelligence,  Surveillance,  and 
Reconnaissance  (ISR)  unmanned  ground  vehicles  (UGV) 

•  Video  analytics  and  tracking  algorithms  to  provide  estimation  of  crowd  size,  allow 
recognition  of  potentially  aggressive  behavior,  and  identify  possible  presence  of 
instigators,  including  tracking  of  personnel  or  objects  of  interest  across  non-overlapping 
video  coverage 

•  Algorithms  to  generate  three-dimensional  (3D)  buildings  and  terrain  automatically  using 
the  images  and  video  streams  collected  by  the  UAV,  UGVs,  and  smartphones  to  provide 
the  most  up-to-date  3D  terrain  and  building  map  of  the  area  of  operation,  which  may  have 
changed  due  to  the  effects  of  natural  disasters  or  collateral  damages 
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C4 


•  Modular  mobile  Command  and  Control  (C2)  and  smart  devices  serve  as  tactical  C4 
systems  to  facilitate  responsive  dissemination,  coordination,  and  execution  in  the  urban 
environment 

•  Fusion  algorithm  to  fuse  structured  reports  from  ground  survey  teams,  other  external  data 
sources,  and  the  fused  information  correlated  and  tagged  to  the  respective  geo-locations 
within  the  3D  terrain  map 

•  Sense  making  algorithms  to  help  identify  patterns  and  detect  anomalies  within  the  fused 
data  sources,  including  identifying  important  social-cultural  relationships  within  the 
distressed  populace,  and  helping  to  predict  potential  outbreaks  of  epidemics  or  social 
unrest 

•  HADR  C2  system  provides  a  private  communication  network  to  link  up  commercial 
smartphones  and  military  radios 

Response 

•  Calibrated  crowd  simulation  models  calibrated  to  represent  the  local  populace  to  help 
planners  and  commanders  study  what-if  scenarios  and  develop  contingency  plans  for 
missions  that  involve  the  local  populace 

•  Dynamic  resource  allocation  decision  support  engines  to  help  match  resources  to  the 
demands  gathered  from  the  operators  on  the  ground.  The  dynamism  arises  from  the 
streaming  of  relief  stockpiles  into  the  area  of  operations  and  the  uncertainty  in  terms  of 
demands  put  forth  by  displaced  personnel 

•  Various  non-lethal  means  (for  example,  Area  Denial  System,  Long-range  Acoustic 
Device)  to  manage  the  civilian  populace,  which  may  potentially  turn  violent 


Operational  Problem 

The  purpose  of  OUECF  is  to  explore  and  demonstrate  technologies  and  capabilities  for  urban 
operations  in  the  context  of  HADR  scenarios.  The  urban  environment  hosts  a  number  of 
operational  challenges.  These  challenges  include,  but  are  not  limited  to  the  following: 

•  Diminished  line  of  sight 

•  Physical  line  of  sight 

•  Communications  line  of  sight 

•  “Dirty  battlefield” 

•  Urban  environments  populated  with  non-combatants 

•  Limited  situational  awareness  at  the  tactical  level 

•  Decreased  reaction  time  at  the  individual  and  small  unit  level 

•  Limited  integration  of  ISR  assets  into  a  common  operational  picture 

OUE  14.2  mainly  focused  on  addressing  the  limited  situational  awareness  at  the  tactical  level 
problem  by  exploring  the  use  of  integrated  ISR  assets  and  applications  designed  to  enhance  the 
warfighters’  situational  awareness  during  operations. 
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OUE  14.2JD  Objectives 

To  support  continued  collaboration,  the  TEC,  FSTD,  and  MINDEF  developed  the  following  3 
main  objectives  for  the  OUE  14.2  TD: 

•  Objective  1:  Demonstrate  interoperability  between  U.S.  and  Singapore  systems  and 
technologies 

•  Objective  2:  Cross-sharing  of  U.S.  and  Singapore  capability  development  efforts 

•  Objective  3:  Address  a  common  operational  problem  shared  by  the  SAF  and 
USPACOM  forces,  conducting  operations  in  urban  environments 

Additionally  the  following  secondary  objectives  were  developed  to  help  focus  efforts  and 
identify  practical  applications  for  OUE  operations: 

•  Support  continued  collaboration  between  the  U.S.  and  Singapore 

•  Collect  feedback  on  OUE  TD  participating  technologies 

•  Collect  feedback  on  hands-on  training  conducted  as  part  of  the  buildup  to  event  execution 

•  Illicit  suggestions  for  future  applications  and  technologies  to  support  HADR  operations 

•  Identify  suggested  improvements  to  existing  applications  and  capabilities 

Technology  Description 

The  following  section  provides  a  brief  description  of  each  of  the  technologies  that  participated  in 
VM14.2.  The  Joint  Non-Lethal  Weapons  (JNLW)  technologies  were  demonstrated  during  the 
event,  but  were  not  part  of  the  overall  integration  or  data  collection  effort. 

Persistent  Ground  Surveillance  System  (PGSS)  Technologies 

Persistent  Ground  Surveillance  System 

(PGSS)  is  a  lighter-than-air,  medium-sized 
tethered  aerostat  that  provides  an  ISR 
capability  for  temporary  and  semi¬ 
permanent  operating  locations  capable  of 
providing  persistent,  integrated,  networked 
and  multi-sensor  information  collection  for 
long  periods  of  time  from  various  altitudes 
to  maximize  line-of-site  and  field-of- 
regard. 
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PGSS  Mobile  Client 

The  PGSS  Mobile  Client  is  an  extension  of 
the  Ageon  ISR  software,  developed 
specifically  for  a  mobile  device  to  provide 
portable  real-time  situational  awareness  to 
the  operator.  The  Mobile  App  leverages  the 
PGSS  ground  control  station's  (GCS) 
network  connection  in  order  to  access  live 
full  motion  video  (FMV)  from  dedicated 
mission  sensors.  The  capability  also 
provides  blue  force  tracking,  geotagging, 
chat,  voice  comms,  and  the  ability  for  the 
user  to  slew  mission  sensors. 


PGSS  4G  LTE  System 

The  4G  LTE  system  is  divided  into  two  main  subsystems.  A  radio  unit  (RU)  and  associated 
antennas,  cabling,  and  mounting  hardware  integrated  onto  the  PGSS  aerostat.  And  the  main  unit 
(MU),  power  unit  (PU),  and  Global  Positioning  System  (GPS)  antenna,  located  at  the  ground 
station.  Communication  between  the  RU  and  the  MU  is  provided  over  a  single-mode  optical 
fiber  that  connects  the  PGSS  to  the  ground  station.  The  system  has  been  licensed  for  15 
concurrent  users  and  should  support  an  aggregate  throughput  of  37  Mbps  download  and  25  Mbps 
upload  for  a  10  MHz  bandwidth  configuration.  For  reduced  bandwidth  configurations,  the 
aggregate  throughput  is  reduced  proportional  to  the  reduction  in  bandwidth. 


AG  LTE 
Smartphone 


Omni-antertna 


xiphos  Radio 

Hare  17 


1000  feet 


X-pho*  Mam 
Unit 


GFE  Vingle- 
nrwde  optical 
fiber 


The  RU  transmits  60  W  output  power.  The  coverage  provided  by  the  system  is  dependent  upon 
the  deployed  height  of  the  aerostat.  At  a  height  of  1000  feet  AGL  and  relatively  flat  terrain,  a 
radius  of  coverage  of  approximately  10  miles  is  predicted.  The  RU  will  operate  in  LTE  Band  17 
with  an  uplink  center  frequency  of  710  MHz  and  a  downlink  center  frequency  of  740  MHz. 
Both  the  uplink  and  downlink  transmissions  will  occupy  a  nominal  either  5  MHz  or  10  MHz 
bandwidth  depending  on  the  network  configuration  under  test. 
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Video  Analytics 

Smart  Video  Analytics  (VA) 

The  Smart  Video  Analytics  software  developed 
by  A*STAR  Institute  for  Infocomm  Research 
(I2R)  uses  electro-optical  (EO),  visible  light 
camera  video  to  detect  and  track  people  and 
identify  suspicious  activities.  Video  feeds  from 
various  sources  can  be  used  as  a  master  camera 
that  detects  and  tracks  people.  The  detections 
are  tracked  geographically  by  the  system  and 
displayed  on  a  map.  The  location  is  used  to 
cross  cue  a  slave  camera.  The  slave  camera 
imagery  is  used  to  identify  suspicious 
characteristics  and  actions. 


DAVION  Applications 

DAVION  Map 

Users  are  able  to  track  each  other’s  movement  on  the  map  once 
they  logon  to  the  system.  Each  user’s  location  will  be 
represented  by  a  blue  drop-pin  displayed  on  the  2D  map.  When 
the  users  move,  the  blue  force  tracking  module  will  reflect  their 
updated  locations  on  the  two  dimensional  (2D)  map  every  30 
sec.  A  blue  circle  represents  the  location  of  the  current  smart 
phone  while  a  red  drop-pin  represents  enemy  force.  Green  drop- 
pin  represents  general  report  and  purple  drop-pin  represents 
point  of  interest  (POI).  The  image  to  the  right  shows  the  current 
user’s  location,  2  blue  forces,  5  red  forces  and  1  general  report. 


DAVION  Augmented  Reality  (AR) 

The  AR  view  superimposes  information  such  as 
locations  of  blue  forces  and  reports  over  live 
video  feeds.  By  using  the  user’s  facing-direction, 
we  are  able  to  superimpose  information  of  blue 
forces,  red  forces,  POI  on  the  video  feedback 
that  is  shown  in  Fig. 2.  A  small  radar  can  be 
found  at  the  top  left  position  as  highlighted  in 
box  1.  This  displays  the  location  of  all  blue 
forces,  red  forces  and  POI  surrounding  the 
mobile  device  of  interest.  It  allows  users  to 
quickly  see  how  many  blue  forces,  red  forces  and 
POI  are  in  the  plan  field  of  view. 
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DAVION  Group  Chat 

This  function  is  similar  to  the  popular  group  chat  app, 
WhatsApp.  The  apps  features  include  one-to-one 
messaging,  one-to-many  messaging,  many-to-many 
messaging  (group  chat),  and  file  sharing.  All  information 
will  be  transmitted  using  a  backend  server. 


lype 


mess, 


here 


Oftimnit 


DAVION  WhiteBoarding 

White  boarding  allows  users  to  draw  free  hand  on  2D  street 
map.  Multiple  users  will  be  able  to  share  their  drawings  in  near 
real  time.  In  lab  conditions,  we  are  able  to  achieve  <4  sec  lag 
for  10  users.  The  full  set  of  features  includes: 

i.  Layer 

ii.  Shapes  &  colours 

iii.  Archiving 

iv.  Undo 


Joint  Non-Lethal  Weapons  (JNLW) 

Pre-emplaced  Electric  Vehicle  Stopper 
(PEVS)  PEVS  is  designed  to  stop  and  disable 
vehicles.  PEVS  has  the  potential  to  support 
multiple  missions  including:  force  protection 
and  vehicle  checkpoint  operations.  This 
technology  is  designed  to  slow  or  arrest 
vehicle  momentum  or  to  be  used  in  concert 
with  a  barrier  or  entanglement  system  to  stop 
vehicles.  The  system  is  a  pre-emplaced,  non- 
intrusive  device  that  provides  an  electrical 
pulse  through  deployed  contacts,  to  shut  down 
power  train  electrical  circuits  or  components. 


OUE  14.2  Final  Report 


Distributed  Sound  and  Light  Array  (DSLA)  - 
Lite  Distributed  Sound  and  Light  Array  (DSLA) 
is  a  non-lethal  acoustical  and  optical  device  that 
provides  hailing  and  warning  capabilities.  Still  in 
the  developmental  stage,  the  DSLA  uses  the 
combined  effects  of  two  integrated  sensory 
stimulators:  a  distributed,  high-output,  phased 
acoustic  array  and  a  distributed,  high-output, 
coherent  (laser)/non-coherent  (bright  white  light) 
optical  array.  As  part  of  the  DSLA's  hailing  and 
warning  capabilities,  the  DSLA's  light  array 
attracts  the  attention  of  the  target,  while  the  sound 
array  conveys  specific  instructions  to  the  target. 
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Locations 


EXECUTION 


Camp  Pendleton  Area  of  Operations  (AO) 

Marine  Corps  Base  Camp  Pendleton  (Figure  1),  the  Corps'  largest  West  Coast  expeditionary 
training  facility,  encompasses  more  than  125,000  acres  of  Southern  California  terrain.  Located 
approximately  38  miles  from  downtown  San  Diego  in  North  County  and  82  miles  south  of  Los 
Angeles,  Camp  Pendleton  has  been  the  largest  employer  in  North  San  Diego  County  for  more 
than  60  years.  Camp  Pendleton  is  one  of  the  Department  of  Defense's  busiest  installations  and 
offers  a  broad  spectrum  of  training  facilities  for  many  active  and  reserve  Marine,  Army  and 
Navy  units,  as  well  as  national,  state  and  local  agencies. 


Figure  1:  Camp  Pendleton  AO 


Piedra  del  Lumbre  (PDL)  Combat  Town  (CBT) 

PDL  Combat  Town  (Figure  2)  is  composed  of  68  containers  making  23  simulated  buildings.  The 
facility  allows  for  non-live  fire  training  in  a  realistic  industrial-looking  environment  that  includes 
guard  houses,  maintenance  shacks,  water  tower,  steam  pipes,  vehicle  gates,  pedestrian  gates,  pry 
doors,  solar-powered  street  lights,  weapons  cache,  and  athletic  field.  PDL  Canyon  Road  runs 
north-south  along  the  east  side  of  PDL  Combat  Town.  Artillery  Firing  Area  Golf  (AFAG) 
borders  the  west  side  of  PDL  Combat  Town  and  was  used  in  the  Joint  Non-Lethal  Weapons 
Directorate  (JNLWD)  scenario  for  the  vehicle  approach  to  the  vehicle  checkpoint  located  on  the 
west  side  of  PDL  CBT  and  non-lethal  weapons  effects/surface  danger  zones.  The  athletic  field 
bleachers  will  was  used  for  the  Tech  Demo  visitors  and  static  display  area  (Figure  3). 
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Figure  2:  PDL  Combat  Town 


Figure  3:  PDL  Combat  Town  Demonstration  Areas 
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Participants 


The  OUE  14.2  event  participants  (Figure  4)  included  military,  government  organizations,  and 
private  technology  companies.  The  OUE  14.2  coordination  and  interface  with  technology 
providers  was  performed  by  the  TEC  working  with  a  team  of  three  MINDEF/SAF  organizations. 


Figure  4:  OUE  14.2  Team  with  Singaporean  Guard  Users 


Technology  Experimentation  Center  (TEC) 

The  TEC,  in  support  of  USPACOM,  coordinated  with  all  OUE  participants,  VM  14.2  U.S. 
forces,  and  U.S.  Tech  Demo  observers;  arranged  logistics  support;  helped  developed  OUE 
scenarios;  and  documented  events  and  participant  feedback  through  briefings  and  this  report. 
The  TEC  is  a  U.S.  Government  consortium  of  technology  and  operational  community  subject 
matter  experts  working  together  to  enable  the  warfighter  by  conducting  technology 
demonstrations,  experiments,  and  assessments  in  relevant  operational  venues  and  environments. 


Singapore  Ministry  of  Defence  (MINDEFj/Singapore  Armed  Forces  (SAF)  Team 

The  MINDEF/SAF  team  consisted  of  military  and  civilian  personnel  from  the  Future  Systems 
and  Technology  Directorate  (FSTD),  Joint  Plans  and  Transformation  Department  (JPTD),  and 
Systems  Integration  Office  (SIO).  FSTD  was  the  coordination  lead  for  the  MINDEF/SAF  team, 
coordinating  among  MINDEF/SAF  and  with  all  Singapore  technology  providers.  JPTD,  as  a 
signatory  on  the  OUE  Collaboration  Framework,  ensured  that  OUE  activities  aligned  with 
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CDWG  intent.  SIO  provided  SAF  warfighter  input  to  OUE  scripted  scenario  development  and 
execution. 

Naval  Air  Systems  Command  (NAVAIR)  Rapid  Reaction/Irregular  Warfare  (RR/IW) 

NAVAIR  is  a  U.S.  technology  provider,  deploying  the  Persistent  Ground  Surveillance  System, 
EO/IR  sensors,  and  common  operating  picture  software.  NAVAIR  provided  an  outdoor  wireless 
mesh  communications  system  that  provided  local  Wi-Fi  across  the  area  of  operations  for  OUE 
technologies.  NAVAIR  Subject  Matter  Experts  (SMEs)  provided  training  to  users,  and  SME 
feedback  on  the  integration  process. 

Agency  for  Science,  Technology  and  Research  (A*STAR)  Institute  for  Infocomm  Research 
Institute  for  Infocomm  Research  (I2R) 

A*STAR  I2R  provided  video  analytics  software  and  video  surveillance  cameras.  A*STAR 
worked  with  the  MINDEF/SAF  and  PGSS  teams  to  install  the  video  cameras  in  and  around  PDL. 
A*STAR  incorporated  video  feeds  from  surveillance  cameras  and  the  NAVAIR  system. 

A*STAR  provided  engineers/scientists  to  perform  various  tasks  including  camera  installation, 
network  configuration  and  integration,  software  tuning,  scripted  scenario  coordination,  and  SME 
feedback. 

Singapore  Technologies  (ST)  Electronics  (Info- Software  Systems) 

ST  Electronics  provided  engineers/scientists  to  incorporate  video  feeds  from  NAVAIR  and 
A*STAR  systems  into  handheld  devices.  ST  provided  hands-on  training  for  OUE  Users  and 
Tech  Demo  observers. 

Alpha  Company,  1st  Battalion,  5th  Marine  Regiment,  1st  Marine  Division 

Alpha  Company  provided  10  users  for  training  and  use  of  the  handsets  during  the  mechanized 
assault,  and  5  users  who  remained  to  execute  vignettes  and  participate  in  DV  Day  (Figure  5). 
Headquartered  at  Marine  Corps  Base  Camp  Pendleton,  Calif.,  1st  Marine  Division  is  a  multi¬ 
role,  expeditionary  ground  combat  force.  The  Division  is  employed  as  the  ground  combat 
element  of  I  Marine  Expeditionary  Force  or  may  provide  task-organized  forces  for  assault 
operations  and  such  operations  as  may  be  directed.  The  1st  Marine  Division  provides  amphibious 
forcible  entry  capability  to  the  naval  expeditionary  force  and  conducts  subsequent  land 
operations  in  any  environment.  The  5th  Marine  Regiment  is  an  infantry  regiment  of  the  United 
States  Marine  Corps.  It  is  the  most  highly  decorated  regiment  in  the  Marine  Corps  and  falls  under 
the  command  of  the  1st  Marine  Division  and  the  I MEF. 


OUE  14.2  Final  Report 


14 


Figure  5:  U.S.  Marine  Users 


Singaporean  Guards 

The  Singaporean  Guard  provided  5  users  that  participated  in  training,  vignettes,  and  the  DV  Day 
demonstration  (Figure  6).  In  the  Singapore  Armed  Forces,  the  Guards  are  an 
elite  infantry  formation  specializing  in  rapid  deployment.  Guards  are  known  as  elite  heliborne 
troopers.  They  are  well  trained  in  heliborne  operations  and  specialist  combat  skills  that  give  them 
an  added  combat  edge.  They  are  proficient  in  heli-rappelling,  heli-landing,  and  other  specialized 
skills  that  allow  them  to  carry  out  heliborne  operations  in  various  terrain,  day  and  night.  Because 
of  their  special  training  they  have  earned  respect  and  status  in  the  Singapore  Armed  Forces.  The 
formation  traces  its  roots  back  to  the  7th  Singapore  Infantry  Brigade  and  has  been  actively 
involved  in  several  National  and  Army  Day  events  since  the  establishment  of  the 
modem  Guards  unit. 


Figure  6:  Singaporean  Guard  Users 
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Data  Sources 


The  TEC  OUE  team  utilized  the  following  data  sources  to  collect  data  and  feedback  on  the  OUE14.2 
TD  event: 

Questionnaires/Surveys 

OUE  user  groups  completed  questionnaires/surveys  designed  primarily  to  gather  feedback  on  the 
OUE  technologies  and  the  integration  process.  Questions  used  a  six-point  rating  scale  ranging 
from  Completely  Disagree  to  Completely  Agree  and  provided  space  for  comments  to  allow  users 
to  explain  their  ratings,  or  to  comment  further.  In  addition,  a  Not  Applicable  (N/A)  choice  was 
available  to  those  users  who  felt  a  particular  question  did  not  apply  to  them.  A  separate  page  of 
the  questionnaire  was  devoted  to  demographic  information. 

Interviews 

When  appropriate,  OUE  user  group  participants  were  asked  to  participate  in  interviews  with  data 
collectors  on  a  noninterference  basis.  Questions  were  designed  to  collect  user  feedback  on  the 
OUE  technologies  in  relations  to  relevant  measures. 

Event  Logs 

Event  Logs  were  used  to  capture  subjective  and  objective  data  during  the  OUE  events.  The  data 
captured  included  performance  data,  timeline,  user  impressions,  SME  observations,  and  the  data 
collectors’  independent  view. 

Photographs 

Data  collectors  captured  photographs  to  support  the  TD.  Data  collectors  ensured  photographs 
remain  unclassified  and  are  approved  for  release  by  the  appropriate  agencies. 

OUE  Schedule 

Table  1  provides  a  summary  of  the  OUE  1.2  TD  events. 


Table  1:  OUE  14.2  TD  Events 


Date 

Location 

Event 

12/5/14 

Oceanside,  CA 

-Receive  PGST  equipment 

12/6/14 

Oceanside  and  PDL 

-Occupy  PDL 
-PGSS  Setup 

12/7/14 

Oceanside  and  PDL 

-PGSS  Setup/VA,  DAVION  Integration 

12/8/14 

Oceanside  and  PDL 

-PGSS  Setup/VA,  DAVION  Integration 

12/9/14 

Oceanside  and  PDL 

-PGSS  Setup/VA,  DAVION  Integration 

12/10/14 

PDL 

-Determine  and  mark  Vignette  routes 
-PGSS  Setup/VA,  DAVION  Integration 
-Data  Collection  and  Testing 
-NLW  Setup  and  Prep 

12/11/14 

PDL 

-Singaporean  Guard  Training  and  Vignettes 
-User  Surveys 

12/12/14 

PDL 

Operations  canceled  due  to  inclement  weather 

12/13/14 

PDL 

-NLW  Setup  and  Prep 

12/14/14 

PDL 

-OPFOR  Training  and  Vignettes 
-User  Surveys 
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-No  NLW  activities 

12/15/14 

PDL 

-Mech  Assault 
-NLW  Training 

12/16/14 

PDL 

-PGSS  Setup/VA,  DAVION  Integration 
-Range  testing  of  4G  LTE 

12/17/14 

PDL 

Operations  canceled  due  to  inclement  weather 

12/18/14 

PDL 

-DV  Day  Rehearsal 
-NLW  Scenarios 

12/19/14 

PDL 

-DV  Day 

-User  Survey  Reviews 
-Equipment  teardown 

12/20/14 

PDL 

-Depart  PDL 

-PGSS  Equipment  pickup 

Data  Collection  Approach 

The  TEC  conducted  OUE  14.2  during  VM  14.2.  VM  14.2  provided  a  bilateral  exercise  venue 
for  collecting  feedback  on  and  demonstrating  the  technologies  participating  in  OUE  14.2.  The 
majority  of  the  information  gathered  to  support  the  OUE  14.2  event  was  objective  data  gathered 
from  event  and  daily  logs,  and  subjective  data  from  the  DAVION  and  PGSS  Mobile  Client  users 
and  Subject  Matter  Expert  (SME)  observations.  User  feedback  and  SME  observations  were 
collected  via  questionnaires  and  interviews. 

The  data  collection  process  consisted  of  three  main  focus  areas;  integration  DAVION 
applications  feedback,  and  PGSS  Mobile  Client  feedback.  As  part  of  OUE  14.2  U.S.  and 
Singaporean  technologies  were  integrated  to  demonstrate  the  combined  capabilities  of  the 
technologies.  The  integration  also  provided  a  learning  environment  that  fostered  information 
sharing  and  collaboration.  The  main  technology  integration  period,  time  set  aside  specifically  to 
integrate  the  technologies  before  user  training,  was  scheduled  for  7-12  December.  Due  to  shifts 
in  user  availability,  users  arrived  early  and  shortened  the  integration  period  to  7-10  December. 
However,  integration  continued  in  a  limited  form  during  training,  as  to  not  negatively  affect  the 
users,  and  in  the  days  following  to  tweak  the  integration  methods  for  maximum  performance.  As 
part  of  the  integration  process  limited  range  testing  was  conducted  on  the  PGSS  4G  LTE 
capability.  To  document  the  entire  integration  process  a  third  party  IT  SME  observed  and 
documented  the  process  to  include  setup,  problems,  resolutions,  and  success. 

The  two  other  focus  areas  of  OUE  14.2  included  collecting  feedback  on  the  DAVION 
applications  and  PGSS  Mobile  Client.  As  part  of  the  event,  the  TEC  team  organized  user  groups 
from  the  Singaporean  Guard  and  the  U.S.  Marines.  Users  from  each  group  were  sorted  into 
DAVION  or  PGSS  users  and  participated  in  training  and  vignettes  designed  to  familiarize  them 
with  the  applications.  The  Singaporean  and  U.S.  teams  each  separately  completed  a  full  day  of 
training,  one  day  of  rehearsal,  and  a  half  day  dedicated  to  a  live  demonstration  during  the  OUE 
1.4  DV  Day.  Users  were  also  provide  ISR  equipped  smartphones  for  use  during  the  mechanized 
assault  conducted  on  15  December  2014.  Throughout  the  event  users  were  provided  surveys  to 
collect  their  feedback  on  the  technologies. 
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General  Data  Collection  Plan 


Tables  2  and  3  describe  the  data  collection  plan  for  the  OUE  VM14.2  TD  event.  Data  was 
collected  using  various  methods  outlined  in  the  data  sources  section  of  this  document.  The 
primary  data  collection  team  consisted  of  the  Lead  Analyst,  a  MINDEF/SAF  team 
representative,  and  the  Information  Technology  (IT)  SME. 


Table  2:  Technology  Integration  (Week  1) 


Event 

Data  Collectors 

Data  Collected 

PGSS 

Setup/PGSS, 

VA,  DAVION 
Integration 

Data  Collection 
and  Testing 

-IT  SME  (Observations, 
diagrams,  and  event  logs) 
-Lead  Analyst  (Support 

IT  SME  as  needed, 

Surveys  and 

Demographics) 

-PGSS,  VA,  DAVION 
teams  (event  logs) 

•  Technical  and  networking  data  on  the  setup  of  the 
PGSS,  VA,  DAVION  systems 

•  Integration  observations 

•  Event  logs  for  any  integration,  interoperability,  or 
reliability  issues 

•  Event  logs  for  LTE  range  testing 

•  SME  Surveys 

•  SME  Demographic  Forms 

Table  3:  Application  Training  and  Vignettes,  Cont.  Integration,  TD  (Week  2) 


Event 

Data  Collectors 

Data  Collected 

Singaporean 

Guard 

Training  and 
Vignettes 

-Lead  Analyst  (Training  and  Vignettes, 
event  logs) 

IT  SME  (Observations,  diagrams,  and  event 
logs) 

-PGSS,  VA,  DAVION  teams  (event  logs) 

•  Guard  User  Surveys 

•  Guard  Demographic  Forms  (during  training) 

•  Observations  on  training,  users,  and  events 

•  Technical,  networking,  and  integration  of  technologies. 

•  Event  logs  for  issues  and/or  range  testing 

U.S.  Marine 
Training  and 
Vignettes 

-Lead  Analyst  (Training  and  Vignettes, 
event  logs) 

IT  SME  (Observations,  diagrams,  and  event 
logs) 

-PGSS,  VA,  DAVION  teams  (event  logs) 

•  Marine  User  Surveys 

•  Marine  Demographic  Forms  (during  training) 

•  Observations  on  training,  users,  and  events 

•  Technical,  networking,  and  integration  of  technologies. 

•  Event  logs  for  issues  and/or  range  testing 

Mech  Assult 

Vignette, 

Training, 

-Lead  Analyst  (Training  and  Vignettes, 
event  logs) 

IT  SME  (Observations,  diagrams,  and  event 
logs) 

-PGSS,  VA,  DAVION  teams  (event  logs) 

•  Observations  on  training,  users,  and  events 

•  Technical,  networking,  and  integration  of  technologies. 

•  Event  logs  for  issues  and/or  range  testing 

DV  Day 
Internal 
Rehearsal 

-Lead  Analyst  (DV  Day  details) 

IT  SME  (Observations,  diagrams,  and  event 
logs) 

-PGSS,  VA,  DAVION  teams  (event  logs) 

•  DV  Day  schedule  and  layout 

•  Technical,  networking,  and  integration  of  technologies. 

•  Event  logs  for  issue  and/or  range  testing 

DV  Day 
Rehearsal 

-Lead  Analyst  (DV  Day  details  and 
observations) 

IT  SME  (Observations,  diagrams,  and  event 
logs) 

-PGSS,  VA,  DAVION  teams  (event  logs) 

•  Observations  on  training,  users,  and  events 

•  DV  Day  schedule  and  layout 

•  Technical,  networking,  and  integration  of  technologies. 

•  Event  logs  for  issues  and/or  range  testing 

DV  Day 

-Lead  Analyst  (DV  Day  details  and 
observations) 

IT  SME  (Observations,  diagrams,  and  event 
logs) 

-PGSS,  VA,  DAVION  teams  (event  logs) 

•  Observations  on  training,  users,  and  events 

•  DV  Day  schedule  and  layout 

•  Technical,  networking,  and  integration  of  technologies. 

•  Event  logs  for  issues  and/or  range  testing 

OUE  14.2  Final  Report 


18 


Scope  and  Limitations 


Due  to  the  nature  of  a  demonstration/exercise  environment,  certain  limitations  impacted  the 
demonstration  and  data  collection.  Participants  attempted  to  use  all  efforts  and  means  to  mitigate 
the  impact  of  limitations  on  the  event.  Table  4  provides  a  summary  of  the  OUE  14.2  limitations. 


Table  4:  OUE  Limitations 


LIMITATION 

IMPACT 

MITIGATION 

The  DAVION  and 
PGSS  Apps  are 
still  in 

development. 

Both  suites  of  applications  had  problems  with 
updating  information  in  real  time.  System 
crashes  were  noted  by  users  and  data 
collectors. 

For  survey  purposes,  users  were 
informed  of  the  developmental 
nature  of  the  technology.  For 
development  purposes,  event  logs 
were  collected  on  reliability  issues 
throughout  the  event. 

The  OUE  event  is 
scenario  based. 

Due  to  time  and  network  constraints, 
scenarios  had  to  be  conducted  within  a  limited 
area.  This  impacted  the  realism  of  some  of 
the  patrol  scenarios,  but  users  were  still  able 
to  utilize  the  applications  as  needed  for 
exposure  and  data  collection. 

Scenarios  were  developed  to 
simulate  realistic/believable  events  to 
ensure  users  were  able  to  provide 
desired  feedback.  Data  collectors 
tailored  scenarios  as  needed  to  help 
support  the  data  collection  process. 

Users  had  a 
limited  amount  of 
time  to  use  the 

Apps  and  form 
opinions. 

The  OUE  team  was  able  to  conduct  one  full 
day  of  training  with  the  Singaporean  Guards, 
and  one  with  the  Marines.  Each  group  also 
participated  in  a  rehearsal  day  and  the  DV 

Day.  One  Marine  was  replaced  and  had  to 
receive  training  on  the  rehearsal  day  but  he 
picked  up  the  applications  quickly  and 
became  our  PGSS  HQ  rep  during  the  DV  Day 
demonstration. 

The  OUE  team  worked  to  try  to  get 
additional  time  with  users.  The  OUE 
team  developed  vignettes  to  help 
expose  the  users  to  each  application 
within  a  relevant  scenario.  The  OUE 
team  provided  for  flexibility  in  the 
schedule  to  help  support 
opportunities  for  additional  time  with 
users. 

OUE  is  operated 
within  a  military 
exercise  which 
could  impact 
scheduled  events 

The  training  for  the  Singaporean  Guards 
occurred  2  days  early.  This  strongly  impacted 
the  integration  timeline  and  resulted  in  some 
features  of  the  PGSS,  DAVION,  and  VA  not 
being  available  for  use  during  training 

The  OUE  team  built  flexibility  into 
the  movement  schedule  to 
accommodate  possible  shifts  in 
exercise  events 

Vignettes  were 
conducted  within  a 
limited  range,  real 
world  operations 
would  likely 
involve  larger 
distances 

The  range  of  the  WiFi  limited  the  area  for 
vignette  operations.  Users  were  still  able  to 
gain  exposure  to  the  applications.  This  was 
ultimately  a  positive  as  our  time  was  limited 
and  we  could  not  afford  using  valuable 
training  time  for  users  to  traverse  longer 
distances. 

Detailed  integration,  networking,  and 
rage  data  was  collected  during  the 
event  to  help  support  future  data 
extrapolation  for  various  ranges 

Potential  for  RF 
interference  on 

WiFi  and  LTE 

No  RF  interference  was  reported,  but  there 
were  issues  with  the  WiFi  dropping  during 
training 

The  OUE  team  will  work  with  the 
base  spectrum  team  and  the 
technologists  to  try  to  limit 
interference 
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Vehicle  Control  Point  (VCP)  Monitoring  and  Response  Scenarios 

The  following  scenarios  were  developed  to  explore  the  capabilities  provided  by  the  integration  of 
the  PGSS,  VA,  and  D AVION  technologies.  The  overall  focus  of  the  vignettes  was  to  highlight 
the  usage  of  the  VA  technology,  DAVION  applications,  and  PGSS  Mobile  Client  for  improved 
situational  awareness  and  incident  management.  The  technologists  and  IT  SMEs  recommended 
that  the  scenarios  be  conducted  using  the  Wi-Fi  capability.  As  a  result  all  vignettes  were  run 
using  the  WiFi  capability. 

Vignette  1:  Monitoring  of  activity  in  vicinity 

The  purpose  of  Vignette  1  was  to  provide  a  training  scenario  where  users  could  apply  the 
applications  offered  by  the  PGSS  Mobile  Client  or  DAVION  to  incident  management  missions 
(Figure  7).  The  PGSS  and  VA  imagery  was  used  to  determine  the  normal  Blue  Forces 
(BFUEFOR)/OPFOR  route.  Once  a  deviation  was  detected  an  alarm  was  sent  to  users.  The 
applications  then  helped  facilitate  real-time  situational  awareness  to  support  users  with  response 
efforts.  This  vignette  was  conducted  outside  (open  road)  of  the  PDF  Combat  town  (urban). 
Originally  the  OUE  team  planned  to  also  run  this  vignette  inside  of  the  combat  town,  but  due  to 
time  constraints,  and  camera  issues  the  focus  was  shifted  to  only  the  open  road  version.  The 
suspicious  items  used  in  this  vignette  were  located  outside  of  the  camera  view.  Tables  5  and  6 
provide  vignette  1  detailed  action  plans  for  PGSS  and  DAVION. 


Vignette  1 .  Monitoring  of  activity  in  vicinity 
(People) _ 


Alarm 


2-man  patrol 


•Deviate  from  patrol  path 
by  walking  off  either  side 
of  the  road 


Entrance 


Patrol  path  /  pQ 


>  Track  persons  cross  cameras 

>  Detect  patrol,  normal,  suspicious  paths 

>  Transmit  alarm 


Vignette  Focus 

•  To  determine  normal  BLUEF  OR'OPF  OR  route 

•  BLUEFOR'OPFOR  deviates,  might  be  m  distress  sabotage 

•  Facilitate  real-time  situational  awareness 

•  Incident  management 

•  Conduct  mission  in  routes  outside  and  inside  of  PDL  town 


Cam  2  Master 


Figure  7:  Vignette  1.  Monitoring  of  activity  in  vicinity 
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Table  5:  Vignette  1  Detailed  Action  Plan  (D AVION) 


Vignette  1:  DAVION  (Patrol) 

Scenario 

Action 

Conducting  usual  patrol  in  PDL  town,  or 
between  PDL  town  and  Aerostat  locations 

•  Patrol  user  will  deviate  from  route  to  the  location  of  2 
suspicious  items 

The  patrol  route  has  been  rigged  with 
security  cameras  and  smart  software  that 
can  detect  changes  in  expected  activity 

•  All  users  receive  an  alarm  that  someone  has  deviated 
from  the  usual  patrol  route 

•  All  users  except  Patrol  user  use  DAVION  Map  to 
determine  if  the  alarm  was  triggered  by  a  friendly  or  an 
intruder 

The  deviation  was  triggered  by  a  friendly 

•  Users  will  contact  Patrol  user  using  the  Chat  feature  to 
determine  reason  for  course  change 

Patrol  user  has  approached  suspicious 
items  and  needs  to  notify  other  users 

•  Patrol  user  responds  to  other  users  with  the  Chat  feature 
and  updates  that  he  is  checking  potential  unknown  items 

Patrol  user  wants  to  know  if  the  suspicious 
items  are  known  items  that  have  been 
documented  previously 

•  Patrol  user  checks  the  Augmented  Reality  capability  and 
determines  one  of  the  two  items  is  known 

One  item  is  not  known  and  needs  to  be 
documented 

•  Take  photo  of  unknown  item  and  document  in  DAVION 
Map  app 

The  suspicious  object  detonates  and  you 
need  to  document  our  escape  route 

•  Use  WhiteBoarding  to  note  safe  meeting  location 

•  Send  Chat  to  notify  users  of  new  WhiteBoarding 
information 

VCP  needs  to  recover  Patrol  user 

•  VCP  users  use  WhiteBoarding  to  alter  the  safe  meeting 
location  and  notifies  the  Patrol  through  Chat 

•  VCP  uses  notes  from  WhiteBoarding  and  Map  to 
navigate  to  Patrol  user  location 

Table  6:  Vignette  1  Detailed  Action  Plan  (PGSS) 


Vignette  1:  PGSS  (Patrol) 

Scenario 

Action 

The  VCP  receives  intel  from 
the  GCS  using  the  Chat  feature 
relating  to  suspicious  items. 

•  VCP  users  will  slew  the  camera  to  the  area  of  suspicion 

•  VCP  user  will  request  the  GCS  zoom  in  on  suspicious  area  using  the 
Chat  feature 

•  VCP  users  tags  the  suspicious  item  location  on  the  map  feature 

Patrol  users  are  conducting 
usual  patrol  between  PDL 
town  and  Aerostat  locations. 

•  VCP  users  send  Chat  message  notifying  the  patrol  to  investigate  and 
photograph  item 

•  Patrol  users  navigate  to  tagged  area  using  streaming  video  and  Map 
features 

The  patrol  has  deployed,  their 
location  needs  to  be  monitored. 

•  VCP  users  monitor  troop  deployment  with  Blue  Force  Tracking 
capability  on  the  Map 

Patrol  locates  the  items. 

•  Patrol  users  take  photos  for  VCP  users  and  posts 

•  Patrol  calls  VCP  with  telephone  capability  to  confirm  identification 
and  ask  for  orders 

One  of  the  suspicious  items 
explodes 

•  Patrol  users  use  the  Map  to  find  a  safe  meeting/recovery  location 

•  Patrol  users  move  to  recovery  location  and  tag  recovery  position 

VCP  sends  response  team  to 
locate  and  recover  patrol 

•  VCP  users  locate  patrols  recovery  location  using  Map  feature,  take 
photo  of  location,  and  tag  photo  for  after  action 
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Vignette  2.  Early  detection  and  stopping  of  approaching  vehicle 

Vignette  2  used  the  PGSS  and  VA  camera  feeds  to  determine  changes  in  the  number  of  vehicle 
passengers  (Figure  8).  Users  used  applications  to  plan  and  navigate  a  search,  and  to  share 
relevant  information.  Additionally,  VA  was  used  to  detect  human  aggression.  The  vehicle 
began  at  the  top  of  the  small  hill  right  before  the  PGSS  location.  Camera  1  was  located  near  the 
top  of  the  hill  before  the  PGSS  location.  The  second  camera  was  located  just  past  the  helium 
tanks  on  the  other  side  of  the  PGSS  area.  The  final  camera  was  located  at  the  entry  point  to  PDL 
Town  near  the  guard  shack.  Tables  7  and  8  provide  vignette  2  detailed  action  plans  for  PGSS 
and  DAVION. 


•  Aggression  detection  using  VA 

•  Facilitate  real-time  situational  awareness 

•  Incident  management 

Figure  8:  Vignette  2.  Early  detection  and  stopping  of  approaching  vehicle 
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Table  7:  Vignette  2  Detailed  Action  Plan  (D AVION) 


Vignette  2:  DAVION  (Vehicle) 

Scenario 

Action 

VA  notices  a  suspicious  vehicle  that  has 
fewer  passengers  than  originally  detected. 

An  alarm  is  sounded  on  the  DAVION  sets 

•  Use  DAVION  Map  app  to  view  triggered  camera 

•  Use  PGSS  camera  to  survey  area 

PGSS  camera  reveals  the  vehicle  has  stopped 
in  a  different  area  closer  to  PDL  town.  The 
dismounted  persons  are  near  the  helium 
truck. 

•  VCP  user  pins  the  vehicle  stop  and  dismounted 
persons  location  in  DAVION  Map  app 

•  Use  WhiteBoarding  to  plan  search  area  where  persons 
dismounted 

•  2  Search  Users  disperse  to  search  for  suspicious 
persons  using  DAVION  Map,  1  remains  at  VCP 

•  2  Vehicle  Users  use  AR  to  locate  vehicle 

Two  search  teams  are  deployed  and  need  to 
be  monitored  by  the  VCP 

•  VCP  user  tracks  searches  with  DAVION  Map 

The  hostiles  initiate  an  attack  on  the  patrol 
once  discovered.  The  VA  cameras  are  able 
to  detect  aggressive  activities 

•  Users  engage  hostiles  with  simulated  aggressive  force 

•  VCP  monitors  the  activity  and  sends  backup 

•  All  users  receive  an  aggressive  activity  alert 

Hostiles  are  captured  and  vehicle  is  secured 

•  Use  Chat  to  relay  enemy  is  captured  and  vehicle  is 
secured 

Documentation  is  requested. 

•  Use  Chat  and  take  and  send  photos  to  the  VCP 

Table  8:  Vignette  2  Detailed  Action  Plan  (PGSS) 


Vignette  2:  PGSS  (Vehicle) 

Scenario 

Action 

GCS  sends  Chat  of  noticed  suspicious 
vehicle  that  has  dropped  off  two 
passengers  and  continued  along  the  road 
towards  PDL  town. 

•  VCP  users  pin  the  location  of  the  dismounted  persons 

•  VCP  users  sends  out  an  alert  using  the  All  Chat  to  all  users 

•  2  Search  Users  disperse  to  search  for  suspicious  persons 

The  vehicle  has  stopped  in  a  different 
area  closer  to  PDL  town 

•  VCP  user  pins,  slews  camera  if  needed,  the  vehicle  stop 
location  in  Map  feature 

•  2  Vehicle  Users  disperse  to  search  for  vehicle,  1  remains  at 
VCP 

Two  search  teams  are  deployed  and  need 
to  be  monitored  by  the  VCP 

•  VCP  user  tracks  searches  with  Blue  Force  Tracking 

The  hostiles  initiate  an  attack  on  the 
patrol  once  discovered. 

•  Users  engage  hostiles  with  simulated  aggressive  force  at 
vehicle 

•  VCP  monitors  the  activity  and  calls  for  backup  using 
telephone  capability  or  Chat 

•  All  users  receive  an  aggressive  activity  alert,  All  Chat 

Hostiles  are  captured  and  vehicle  is 
secured 

•  Use  Telephone/Chat  to  relay  enemy  is  captured  and 
vehicle  is  secured 

Documentation  is  requested. 

•  Users  take  photos  and  tag  the  images 

Vignette  3:  Mech  Assault  Focus 

The  focus  of  Vignette  3  was  to  provide  users  with  exposure  to  potential  uses  of  PGSS,  VA,  and 
DAVION  technologies  within  a  mechanized  assault  scenario.  Users  simulated  that  the  Mech 
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Assault  was  occurring  and  were  asked  to  work  through  the  vignettes  on  their  own  to  try  to  gain 
familiarity  with  the  applications  within  the  provided  context.  The  users  were  provided  a  vignette 
focusing  on  using  the  applications  as  an  attacker  and  a  vignette  for  use  as  a  defender.  Tables  9, 
10,  11,  and  12  provide  vignette  3  detailed  action  plans  for  PGSS  and  DAVION  defenders  and 
attackers. 

Table  9:  Vignette  3  Mech  Assault  Defenders  (DAVION) 


Vignette  3:  DAVION  (Mech  Assault)  (Defenders) 


Scenario 

Action 

Cameras  are  dispersed  around  PDL  combat  town 
and  UAVs  (Aerostat)  are  patrolling  the  area. 

•  Use  the  video  feeds  to  detect  the  number  of 
incoming  vehicles  and  gain  situational  awareness 

The  enemy  is  using  an  unanticipated  approach 

•  Use  WhiteBoarding  to  adjust  the  response  plan 

•  Tag  response  positions  A,  B,  C  on  DAVION 

Map 

•  Deploying  Users  use  AR  to  navigate  to 
positions  A,  B,  C 

Troops  have  deployed  based  on  the  new  response 
positions.  Their  locations  need  to  be  monitored. 

•  VCP  users  monitor  troop  deployment  using 
DAVION  Map 

Need  confirmation  of  arrival  at  response  position 

•  Deploying  Users  send  Chat  notifying  VCP  of 
arrival  at  response  position 

•  VCP  confirms  correct  position,  and  acknowledges 
arrival 

Enemy  forces  have  entered  PDL  town 

•  All  users  receive  an  alert  the  enemy  has  entered 
PDL  town 

Table  10:  Vignette  3  Mech  Assualt  Attackers  (DAVION) 


Vignette  3:  DAVION  (Mech  Assault)  (Attackers) 


Scenario 

Action 

UAVs  (Aerostat)  are  patrolling  the  area  to  help 
support  the  assault. 

•  Use  the  video  feeds  observe  PDL  Combat  town 
and  relay  any  useful  info  using  the  Chat  and  Map 
tagging  capabilities 

Based  on  the  UAV  info  a  new  assault  approach  is 
needed. 

•  Use  the  Map  tagging  feature  to  drop  attack 
positions  A,  B,  C 

•  Deploying  Users  use  map  to  navigate  to  pinned 
positions 

Troops  have  deployed  based  on  the  new  response 
positions.  Their  locations  need  to  be  monitored. 

•  VCP  users  monitor  troop  deployment  using 
DAVION  Map 

Need  confirmation  of  arrival  at  attack  position 

•  Deploying  Users  send  Chat  notifying  VCP  of 
arrival  at  response  position 

•  VCP  confirms  correct  position,  and  acknowledges 
arrival 

The  town  has  been  secured.  Images  of  PDL  town 
are  requested  by  HQ 

•  Take  photos  of  locations  in  PDL  and  tag  on  map 
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Table  11:  Vignette  3  Mech  Assualt  Defenders  (PGSS) 


Vignette  3:  PGSS  (Mech  Assualt)  (Defenders) 

Scenario 

Action 

Cameras  are  dispersed  around  PDL  combat  town 
and  UAVs  (Aerostat)  are  patrolling  the  area. 

•  Use  the  video  feeds  to  detect  the  number  of 
incoming  vehicles  and  gain  situational  awareness 

The  enemy  is  using  an  unanticipated  approach 

•  Use  Map  to  adjust  the  response  plan 

•  Tag  response  positions  A,  B,  C 

•  Deploying  Users  use  Map  to  navigate  to 
positions  A,  B,  C 

Troops  have  deployed  based  on  the  new  response 
positions.  Their  locations  need  to  be  monitored. 

•  VCP  users  monitor  troop  deployment  using  Map 

Need  confirmation  of  arrival  at  response  position 

•  Deploying  Users  send  Chat  or  telephone  call 
notifying  VCP  of  arrival  at  response  position 

•  VCP  confirms  correct  position,  and  acknowledges 
arrival  using  Chat/Call 

Enemy  forces  have  entered  PDL  town 

•  All  users  receive  an  alert  the  enemy  has  entered 

PDL  town 

Table  12:  Vignette  3  Mech  Assualt  Attackers  (PGSS) 


Vignette  3:  PGSS  (Mech  Assualt)  (Attackers) 

Scenario 

Action 

UAVs  (Aerostat)  are  patrolling  the  area  to  help 
support  the  assault. 

•  Use  the  video  feeds  observe  PDL  Combat  town 
and  relay  any  useful  info  using  the  Chat  and  Map 
tagging  capabilities 

Based  on  the  UAV  info  a  new  assault  approach  is 
needed. 

•  Use  the  tagging  feature  to  drop  attack  positions  A, 

B.  C 

•  Deploying  Users  use  map  to  navigate  to  pinned 
positions 

Troops  have  deployed  based  on  the  new  response 
positions.  Their  locations  need  to  be  monitored. 

•  VCP  users  monitor  troop  deployment  using  Map 

Need  confirmation  of  arrival  at  attack  position 

•  Deploying  Users  send  Chat  notifying  VCP  of 
arrival  at  response  position 

•  VCP  confirms  correct  position,  and  acknowledges 
arrival 

The  town  has  been  secured.  Images  of  PDL  town 
are  requested  by  HQ 

•  Take  photos  of  locations  in  PDL  and  tag  on  map 

DV  Day  Vignette 

After  completing  separate  training  days  using  vignettes  1-3  outlined  above,  the  Singaporean 
Guards  and  U.S.  Marines  returned  on  18  December  to  rehearse  vignettes  for  demonstration 
during  DV  Day.  It  was  determined  Vignette  2  provided  the  best  demonstration  opportunity.  The 
vignette  was  slightly  altered  to  integrate  the  use  of  both  the  DAVION  and  PGSS  technologies. 

To  help  provide  users  with  an  engaging  demonstration  monitors  were  setup  in  the  demonstration 
area  to  provide  the  visitors  an  on  screen  display  of  the  PGSS  GCS  aerial  view,  the  HQ/VCP 
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Users  DAVION  handset,  and  the  HQ/VC P  users  PGSS  handset.  Table  13  provides  the  detailed 
action  plan  for  the  DV  Day  vignette. 


Table  13:  DV  Day  Vignette 


DAVION  and  PGSS  DV  Day  Vignette 

Scenario 

DAVION 

PGSS 

VA  cameras 
notice  a 
suspicious 
vehicle  that 
has  dropped 
off 

passengers 
and  continued 
along  the 
road  towards 
PDL  town. 

•  All  Users  receive  an  alert 

•  VCP  User  uses  Whiteboarding  to  draws 
search  route  and  Chat  to  tell  Helium 

Team  Users  to  deploy  [1  pax  detected  at 
tanks  T1  deploy] 

•  Helium  Team  Users  deploy  to  helium 
tanks  to  find  dismounted  person(s),  when 
team  arrives  they  will  apprehend 
dismounted  person(s)  without  aggression. 
Tag  the  location  of  the  incident.  Send 
back  a  photo  and  chat  message  indicating 
capture  using  Chat  All  [1  pax  captured] 

•  When  DAVION  receives  alert,  VCP 
PGSS  User  slews  camera  to  helium 
tank  area 

•  Monitor  Blue  Force  deployment  on 
Map  to  helium  tank  area 

The  vehicle 
has  stopped 
at  the  guard 
shack  in  front 
of  PDL  town 

•  Vehicle  Team  will  approach  vehicle  and 
ask  driver  questions  regarding  his  purpose 
for  being  in  the  area.  Team  becomes 
suspicious  and  asks  passengers  to  exit  the 
vehicle. 

•  Vehicle  Team  will  approach  vehicle 
and  ask  driver  questions  regarding  his 
purpose  for  being  in  the  area.  Team 
becomes  suspicious  and  asks 
passengers  to  exit  the  vehicle 

Upon  exiting 
the  vehicle 
the  driver 

starts  an 

altercation 
with  a 
member  of 
the  Vehicle 
Team. 

•  All  Users  receive  an  aggression  detection 
alert 

•  Vehicle  User  engages  passenger  in  fight 
and  eventually  subdues  driver 

•  VCP  User,  when  driver  exits  the 
vehicle  slew  camera  to  vehicle  area 

•  VCP  User  use  Chat  to  request  GCS 
zoom  in  on  altercation 

•  Vehicle  User  use  Telephone  Call 
feature  to  report  back  to  VCP  once 
driver  is  detained  [2  pax  detained] 

All  persons 
of  interest 
apprehended 

•  VCP  sends  a  Chat  All  to  return  to  base 

•  Take  all  persons  of  interest  to  guard  shack 

•  VCP  replies  in  call  to  return  to  base 
with  apprehended  person. 

•  Take  person  of  interest  to  guard  shack 

JNLW  Demonstration 

During  the  OUE  event  JNLW  demonstrated  two  non-lethal  technologies.  The  JNLW 
demonstration  main  operating  window  was  from  12/10/14  to  12/20/14.  During  this  time  the 
JNLW  team  trained  users  on  the  use  of  the  PEVS  and  the  DSLA  -Lite.  The  team  utilized  pre 
designed  vignettes  to  provide  training  scenarios  and  demonstrate  the  participating  technologies 
(Ligure  9). 

JNLW  Vehicle  Stopping  Vignettes 

•  Vehicle  Profiles 

1.  Past/Constant:  Vehicle  approaches  VCP  at  25  mph  (doesn't  slow  or  stop) 

2.  Past/Erratic:  Vehicle  approaches  VCP  at  25  mph  (stops,  starts,  brakes,  swerves) 

3.  Slow/Constant:  Vehicle  approaches  VCP  at  10  mph  (doesn't  slow  or  stop) 
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4.  Slow/Erratic:  Vehicle  approaches  VCP  at  10  mph  (stops,  starts,  brakes,  swerves) 

•  Run  distance  is  approx.  750  meters  from  when  car  is  initially  detected  until  it  hits  the 
VCP  gate 

•  We  expect  all  NLWs  engagements  to  take  place  between  750  and  200meters 

•  At  25  mph  these  engagements  will  only  last  50-55  seconds  before  the  cars  are  shot  and 
killed 
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Figure  9:  Vehicle  Stopping  Demo  Layout 


Mechanized  Assault 

On  December  19,  2014,  as  part  of  Valiant  Mark  14.2,  a  mechanized  assault  was  conducted  in 
various  locations  on  Camp  Pendleton.  PDL  Combat  Town  was  one  of  the  defending  location 
during  the  assault.  The  town  was  occupied  by  two  squads  of  the  1/5  Alpha  Company  U.S. 
Marines.  A  range  of  vehicles  surrounded  the  town  in  under  one  hour,  mid-moming,  on  the  19th 
(Figure  10). 


Figure  10:  PDL  Town  Mechanized  Assault 
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As  part  of  the  OUE  14.2  technology  demonstration  and  field  experimentation  10  of  the 
defending  U.S.  Marines  were  provided  an  hour  long  training  session  on  the  participating 
technologies.  Five  of  those  trained  completed  a  full  day  of  vignettes  to  help  better  expose  them 
to  the  technologies  and  their  capabilities.  Vignettes  were  developed  specifically  to  help  the  users 
determine  potential  uses  for  the  technologies  during  the  mechanized  assault.  After  the  full  day  of 
training  and  vignettes,  those  5  Marines  were  able  to  go  back  to  their  team  and  provide  more 
insight  on  the  mobile  application  technologies.  In  the  days  leading  up  to  the  mechanized  assault 
the  defending  Marines  were  able  to  employ  other  PGSS  technologies  to  help  plan  their  response 
(Figure  11). 


Figure  11:  U.S.  Marines  Tour  PGSS  GCS  (Left)  and  Platoon  Commander  uses  the  PGSS 

COP  to  help  plan  a  response  (Right) 


Users  provided  a  lot  of  positive  feedback  regarding  the  usefulness  of  the  OUE  14.2  technologies 
during  the  mechanized  assault  and  similar  scenarios.  The  users  were  able  to  communicate 
during  the  assault  without  giving  away  their  positions,  and  monitor  the  approach  of  the  attacking 
force.  User  comments  relating  to  the  use  of  the  OUE  14.2  technologies  during  the  mechanized 
assault  can  be  found  in  the  DAVION  and  PGSS  User  Training  and  Vignettes  section  of  this 
report. 

Distinguished  Visitors  Day 

The  OUE  team  hosted  a  Science  &  Technology  (S&T)  DV  Day  at  PDL  Combat  Town,  Camp 
Pendleton  on  19  December  2014  (Figures  12,  13,  14,  15,  16).  The  purpose  of  the  event  was  to 
highlight  S&T  projects  as  part  of  an  effort  to  promote  bilateral  S&T  collaboration  between  the 
MINDEF/SAF  and  U.S.  PACOM.  The  event  played  a  key  role  in  highlighting  the  MINDEF  and 
PACOM  collaboration  on  technology  experimentation.  The  S&T  Distinguished  Visitors  Day 
consisted  of  technology  briefs  and  a  scenario  based  technical  demonstration  of  new  and 
emerging  technologies  that  were  currently  engaged  in  OUE  14.2. 

Over  60  guests  attended  the  OUE  14.2  DV  Day  event.  As  the  guests  arrived  they  were  lead  into 
a  covered  pavilion  area  with  bleacher  seating.  Displayed  along  one  side  of  the  area  were  posters 
providing  information  on  each  of  the  OUE  technologies,  and  aspects  of  the  integration  process. 
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Additionally,  a  trifold  providing  a  summary  of  the  OUE  14.2  event  and  technology  information 
was  provided  for  attendees  to  read  and  take  with  them  upon  leaving  the  demonstration. 

After  the  opening  comments  from  the  JNLW  and  OUE  teams  the  DV  Day  attendees  were  asked 
to  observe  a  live  action  scenario  involving  the  PGSS  aerostat,  PGSS  Mobile  Client,  VA,  and 
DAVION  applications.  A  small  mobile  COP  was  setup  in  the  pavilion  area  that  provided 
attendees  a  view  of  the  overall  aerostat  view  and  the  views  on  each  of  the  handheld  application 
platforms  being  operated  by  the  Singaporean  Guard  and  U.S.  Marine  users. 

During  the  live  action  scenario  the  Singaporean  Guards  and  U.S.  Marines  utilized  all  of  the 
participating  technologies  to  demonstrate  the  communications,  information  sharing,  situational 
awareness,  and  aggression  detection  features  of  the  technologies.  The  DV  Day  participants  were 
very  impressed  with  the  capabilities  demonstrated  and  provided  positive  feedback  on  the  value 
of  the  communications  features,  aggression  detection,  and  PGSS  aerostat.  Additional  questions 
regarding  aerostat  payloads  and  implementation  of  the  VA  software  were  asked  by  attendees  and 
addressed  by  the  briefing  technologists. 


Figure  12:  DV  Day  Static  Displays  (Left)  and  Demonstration  Day  Mobile  COP  (Right) 


The  following  distinguished  visitors  participated  in  the  OUE  14.2  DV  Day: 

•  Major  General  Lawrence  D.  Nicholson,  Commanding  General,  1st  Marine  Division 

•  COL  Lim  Siong  Tiong,  Head  Concept  Generation  Group,  Future  Systems  and 
Technology  Directorate 

•  COL  Tan  Cheng  Kwee,  Commander  7  Singapore  Infantry  Brigade  (7  SIB) 

•  Sergeant  Major  David  L.  Jobe,  Sergeant  Major,  1st  Marine  Division 

•  MWO  (Master  Warrant  Officer)  Sanjee  Singh,  Brigade  Sergeant  Major,  7  SIB 

•  LTC  Fan  Man  Poh,  Commanding  Officer,  Guards  Battalion 

•  Capt  Learlin  J.  Lejeune  III,  Alpha  Company  Commander,  1st  Battalion,  5th  Marines 

•  Capt  Ryan  Szabo,  S-2,  1st  Battalion,  5th  Marines, 

•  IstLt  David  Deal,  Platoon  Commander,  Alpha  Company,  1st  Battalion,  5th  Marines 

•  Paul  Funora  of  NAVAIR  (GS-15) 
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Figure  13:  DV  Day  Guests  and  Organizers  Mingle  Before  Opening  Remarks 


Figure  14:  DV  Day  Guests  Receive  Opening  Remarks  from  the  OUE  and  JNLW  Teams 


Figure  15:  DV  Day  Visitors  View  a  Live  Demonstration  of  OUE  Technologies 
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Figure  16:  MG  Nicholsan,  COL  Lim  Siong  Tiong,  and  other  DV  Day  Visitors  Receive 

Briefs  on  OUE  14.2  Technologies 


User  and  SME  Demographics 

This  section  provides  a  summary  of  the  demographic  information  collected  on  the  Singaporean 
Guard  and  U.S.  Marine  users,  and  the  technology  integration  SMEs.  User  demographic 
summaries  can  be  found  in  tables  14  and  15. 

DAVION  and  PGSS  Mobile  Client  User  Demographics 

13  Total  Users  provided  survey  feedback.  Additional  users  were  trained  by  the  surveyed  users  to 
use  the  DAVION  and  PGSS  Mobile  Applications  during  the  Mech  Assault  but  those  users  were 
not  available  to  fill  out  surveys.  Additionally,  some  of  the  original  U.S.  Marines  were  not 
available  for  the  DV  Day  rehearsal  and  demonstration.  Those  marines  were  replaced  with  new 
trainees  who  were  quickly  trained  by  the  technologist  and  subsequently  integrated  into  the 
vignette  rehearsals. 

US  Urban  Environment  Experience 

IRAQ  deployment  from  2008-2009  for  stability  operations  in  A1  Anbar  province. 

Afghanistan  Deployment  in  Helmand  province  2010-2011 

31st  MEU  deployment  2011-2012 

UDP  deployment  to  Australia  2013-2014 

Training  in  MOUNT  environments,  Urban  Combat  Towns,  including  annual  training 

Singaporean  Urban  Environment  Experience 

Annual  Urban  Environment  Training 

US  HADR  Experience 

None 

Singaporean  HADR  Experience 

Local  peacekeeping  exercises 
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*  All  users  noted  that  they  owned  smart  phones  and  were  very  comfortable  using  smart  phone 
applications. 


Table  14:  User  Demographics  by  Technology 


DAVION  Users 

PGSS  Users 

Total  Users 

Singaporean  Guards 

3 

3 

5* 

U.S.  Marines 

4 

4 

8 

*Due  to  changes  in  the  Mech  Assult  and  demonstration  requirements  one  Singaporean  user  was  trained  on  both  systems  and 
provided  feedback  on  each  of  the  systems. 


Table  15:  User  Experience  Demographics 


Average  Years  in 
the  Military 

MOS  Represented 

Average  Years  in 
MOS 

Singaporean  Guards 

3.5 

Infantry  Specialist 
Infantry  Rifleman 

NA 

U.S.  Marines 

3.1 

Machine  Gunner 
Rifleman 

2.9 

SME  Demographics 

A  total  of  eighteen  surveys  were  completed  by  technologies  participating  in  the  integration 
process  during  OUE  14.2.  The  respondents  represented  A-STAR  and  ST  Electronics,  DRG,  and 
the  JHU-APL  and  PGSS  team.  Many  of  the  respondents  had  previous  experience  with  the 
participating  technologies  and  with  previous  OUE  events. 

SME  Relevant  Experience 

Afghanistan  deployments  with  PGSS 

John  Hopkins  University  Applied  Physics  Lab 

Previous  OUE  events 

Designers,  IT,  Field,  Integrators,  Operator 
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PGSS/VA/DAVION  INTEGRATION  RESULTS 

This  section  provides  a  summary  of  the  overall  integration  process  of  OUE  14.2  technologies 
during  VM  14.2,  including  difficulties  and  successes,  the  objectives  and  measure  used  to  collect 
data,  and  feedback  collected  from  users  and  SMEs.  The  observations  collected  in  this  section  of 
the  report  were  recorded  by  an  independent  IT  SME  who  was  not  affiliated  with  any  of  the 
participating  technologies.  A  diagram  of  the  final  technology  integration  setup  can  be  found  in 
Figure  23. 

During  OUE  14.2  the  PGSS  system  and  mobile  client,  Visual  Analytics  Software,  and  DAVION 
applications  were  demonstrated  in  and  around  PDL  Combat  town  with  users  from  Singapore 
Alined  Forces  and  the  U.S.  Marines. 

Basic  Integration  Setup 

The  PGSS  System  utilizes  an  aerostat  as  the  central  hub  of  data  dissemination  (Figure  17).  The 
Aerostat  deployed  a  single  camera  that  provided  an  aerial  view  of  the  area  and  zooming 
capability.  Additionally,  seven  VA  cameras  were  deployed  to  provide  coverage  for  the 
demonstration  of  the  change  and  aggression  detection  capabilities  of  the  system.  The  Aerostat 
camera  video  feeds  flowed  through  the  attached  single  mode  fiber  tether  to  the  servers  inside  the 
GCS  where  the  feeds  were  processed.  From  there,  if  the  phones  were  using  LTE,  the  data  was 
sent  back  up  the  tether,  and  broadcast  to  the  smartphones  on  the  ground.  If  the  phones  were  using 
Wi-Fi,  the  feed  flowed  through  the  wired  and  wireless  network  to  the  smartphones.  The  ground 
based  camera  video  feeds  were  transmitted  via  a  wireless  network  device  to  the  to  the  nearest 
Wi-Fi  access  point  and  through  the  wireless  and  wired  network  back  to  the  Singaporean  servers 
connected  to  the  GCS,  finally  following  the  same  path  as  the  FTE. 


Figure  17:  PGSS  Aerostat  (Left)  and  PGSS  Aerostat  Camera  Setup  (Right) 


OUE  utilized  a  backhaul  internet  connection.  This  connection  originated  at  the  GCS  and  allowed 
the  smartphone  users  on  the  ground  to  make  calls  outside  the  local  network  to  the  rest  of  the 
world  utilizing  4G  FTE  service  provided  by  Verizon.  A  large  antenna  on  top  of  the  GCS 
provided  for  a  dependable  uplink  despite  the  remote  location. 
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To  facilitate  the  integration  of  the  PGSS  and  Singaporean  technologies,  the  PGSS  team  provided 
network  connectivity  and  static  IP  addresses  (.210  -  .240)  for  use  by  the  Singaporean 
technologies.  The  equipment  provided  by  the  Singaporean  team  included: 

•  Seven  AXIS  P5532  PTZ  dome  cameras 

•  PTZ  (Pan  Tilt  Zoom)  security  cameras  rated  for  outdoor  use 

•  10/100  Ethernet  Port,  29x  optical  zoom,  H.264,  Maximum  720x480  resolution 

•  Most  cameras  had  both  a  wired  10/100  Ethernet  and  wireless  connection  available; 
the  final  configuration  for  the  demonstration  consisted  of  five  wired  cameras  and  two 
wireless. 

•  Each  camera  had  an  available  wireless  connection: 

(1)  Outdoor  Wireless  Network  Adapter  Model:  TP-Link  TL-WA7510N 

(2)  Wireless-N,  5Ghz,  150Mbps 

•  The  cameras  and  network  adapters  were  powered  using  PoE  power  supplies  and 
generator(s) 

•  Each  camera  and  network  adapter  was  mounted  on  lx  Hercules  Gear  Up  Speaker 
Stand 

(1)  Cameras  1  and  2  were  mounted  together  on  a  single  stand 

•  One  Asus  AC  1900  gigabit  router,  wired  to  the  main  PGSS  router,  providing  ports  for: 

•  Four  Dell  laptops  for  video  analysis  /  monitoring  /  configuration  purposes 

•  One  Dell  laptop  located  in  the  GCS  and  wired  directly  to  the  PGSS  router. 

•  Smartphones  (approximately  5-10  phones,  using  LTE  or  Wireless-N) 

•  Several  large  directional  5ghz  antennas,  that  were  not  used  in  the  demonstration 

Additional  network  hardware  provided  by  PGSS  to  support  DAVION  network  connectivity 
included: 

•  One  layer  2  gigabit  switch,  providing  wired  connectivity  to  cameras  1  &  2  over  a  single 
Ethernet  uplink  cable  back  to  the  GCS. 

•  Three  Cisco  Aironet  1552  Wireless-N  Access  Points 

•  One  inside  PDL  combat  town,  one  outside  PDL  combat  town,  and  one  at  the  GCS 

•  Configured  in  mesh  mode  to  directly  communicate  with  each  other 

•  Wireless-N  network  (both  2.4ghz  and  5ghz)  for  cameras  and  smartphones 

VA  Camera  Setup 

The  VA  camera  team  worked  throughout  the  OUE  14.2  event  to  ensure  a  successful  integration 
of  their  technology  with  the  other  participating  technologies.  SMEs  diligently  tweaked  camera 
locations  and  the  VA  software,  conducting  daily  tests  to  optimally  train  the  VA  software  training 
and  demonstrations  (Figure  18). 
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Figure  18:  VA  Camera  Placement,  Programming,  and  Troubleshooting 


Three  AXIS  P5532  IP  Cameras  (#5-7)  were  deployed  within  PDL  Combat  Town  to  support  the 
data  collection  effort  and  the  Mechanized  Assault  on  the  15th  December  (Figure  19). 

•  Video  streams  from  each  camera  was  processed  to  determine  the  GPS  coordinate  of  each 
human  detected.  Coordinates  were  recorded  in  the  database  for  consumption/retrieval  via 
the  D AVION  handsets. 

•  Each  camera  was  mounted  on  a  stand  located  on  the  building  roofs,  overlooking  the 
combat  town  to  minimize  obstructions. 

•  Each  camera  was  pre-calibrated  and  configured  for  a  predefined  camera  view. 

•  Each  camera  served  a  specific  role  in  the  vignettes,  providing  feeds  for  the  video 
analytics  module  to  perform  initial  people  count  and  vehicle  tracking  for  the  2nd  vignette. 

Four  AXIS  P5532  IP  Cameras  (#1-4)  were  deployed  between  the  PDL  Combat  Town  and  PGSS 
Ground  Control  Station  to  support  vignettes  and  the  Mechanized  Assault  (Figure  20). 

•  The  video  analytics  module  operating  on  camera  1  supported  the  1st  and  2nd  vignettes. 

•  In  the  1st  vignette,  the  cameras  learned  the  ‘normal  path’  of  patrol  executed  by  a  foot 
patrol  and  alerted  the  DAVION  handsets  upon  detecting  deviation  from  this  normal  path. 
In  the  2nd  vignette,  the  cameras  performed  a  secondary  count  of  the  people  in  the  vehicle 
and  alerted  the  DAVION  handsets  upon  detecting  a  discrepancy  with  the  initial  count. 

•  Camera  3  was  situated  closest  to  the  Vehicle  Control  Point  (VCP)  and  its  analytics 
module  performed  aggression  detection  during  the  2nd  vignette. 

•  All  four  cameras  performed  vehicle  detection,  tracking,  and  counting  during  the 
Mechanized  Assault 
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Figure  19:  PLD  Combat  Town  Camera  Placement 


Figure  20:  Camera  Placement  in  PGSS  Area 
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Evolution  of  the  Integration  Process 


Wireless-N 

The  Wireless-N  network  was  a  work  in  progress  throughout  most  of  the  two  week  period. 
Providing  both  2.4  GHz  and  5  GHz  frequencies,  there  was  a  single  SSID  used  for  the  entire 
network.  The  Cisco  APs  were  moved  several  times  until  optimal  locations  were  found. 

Originally  there  was  a  Cisco  AP  mounted  on  the  aerostat,  but  once  the  aerostat  was  raised  the 
signal  quickly  vanished.  This  is  most  likely  due  to  the  antennas  used  on  the  Cisco  AP,  which 
pointed  outward  (parallel  to  the  horizon),  rather  than  downward  to  the  ground,  and  the  1000’ 
height  of  the  aerostat  when  raised. 

The  original  plan  integration  plan  was  to  connect  all  of  the  cameras  on  the  ground  to  the  network 
via  Wireless-N.  However,  during  the  integration  process  it  was  determined  that  the  AXIS 
Cameras  and  D AVION  server  software  do  not  handle  packet  loss  well.  Two  dropped  packets  in  a 
row  meant  loss  of  the  video  feed  at  the  server  for  up  to  20  seconds;  which  resulted  in  a  failed  test 
in  our  scenarios.  As  a  result,  Ethernet  was  run  to  nearly  every  camera  location  and  the  majority 
of  the  cameras  were  wired  for  the  demonstration.  However,  due  to  the  physical  limitation  on 
length  of  Ethernet  cable  of  100m,  two  of  the  cameras  (3  and  5)  were  wireless.  These  cameras  had 
a  direct  line  of  sight  to  a  Cisco  Access  Points  so  there  were  no  issues  with  these  cameras  during 
the  demonstration  despite  the  range  (especially  Camera  5,  which  was  382  feet  away  from  the 
switch  in  PDL  Combat  Town). 

Power  over  Ethernet  (PoE) 

After  discovering  the  Axis  cameras  were  PoE  capable,  each  was  wired  and  powered  from  the 
Cisco  PoE  switch  located  in  PDL  Combat  Town.  It  was  hypothesized  that  this  would  eliminate 
their  dependence  on  Wireless-N  for  their  network  connectivity  as  well  as  the  need  for  small 
generators  at  each  location.  However,  after  failing  to  power  an  Axis  camera  from  the  switch  we 
determined  that  the  Axis  camera  is  not  compliant  with  802. 3af,  the  standard  Power  over  Ethernet 
that  can  supply  up  to  15  watts  (and  what  the  Cisco  switch  is  capable  of).  The  cameras  were 
compliant  with  the  IEEE  802. 3at;  this  is  known  as  High  Power  over  Ethernet,  which  can  go  as 
high  as  30  watts.  The  switch  wasn’t  able  to  provide  enough  power  to  the  camera,  so  generators 
were  ultimately  used  to  power  each  camera. 

Inputting  the  aerostat  camera  feed  into  DAVION 

The  PGSS  camera  feed  streamed  to  the  Video  Analytics  Milestone  VMS  for  real-time 
processing.  During  the  mechanized  assault  and  2nd  vignette,  the  PGSS  stream  was  processed  for 
vehicular  detection,  tracking  and  counting.  In  the  1st  vignette  the  PGSS  stream  was  processed  for 
human  detection  and  tracking. 

There  were  two  parts  to  the  video  feed  from  the  PGSS  camera.  The  first  was  the  HD  video  feed 
from  the  MX  15  camera  itself,  and  the  second  was  the  telemetry  data  which  provides  metadata 
regarding  camera  direction,  modes,  GPS  coordinates,  zoom  levels,  etc.  This  metadata  was 
encoded  in  a  proprietary  KLV  (Key-Length- Value)  format  that  the  DAVION  system  could  not 
process.  The  Aegon  server  expert  was  able  to  encode  the  metadata  using  a  different  format 
known  as  JSON  (Javascript  Object  Notation),  which  is  a  format  compatible  with  DAVION 
(Figure  21). 
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Figure  21:  DAVION  and  Aegon  Technologist  Integrating  the  Aerostat  Feed  into  DAVION 
Inputting  the  Ground  camera  feeds  into  Aegon 

Multiple  issues  were  encountered  while  attempting  to  get  the  video  feed  from  the  ground 
cameras  into  the  Aegon  system  so  they  could  be  sent  to  the  PGSS  Smartphones.  Multiple  video 
profiles  were  tested  until  finally  the  H.264  pass-through  profile  in  Aegon  was  able  to  import  the 
raw  feed  directly  from  the  camera;  attempting  to  input  the  processed  RTSP  feed  from  the  VA 
server  continually  crashed  the  Aegon  server.  Also,  attempting  to  use  any  other  video  profile 
resulted  in  the  Windows  process  cycling  on  and  off  repeatedly,  which  would  raise  the  memory 
usage  to  alarming  levels,  and  the  process  had  to  be  stopped  from  the  Windows  Task  Manager;  it 
could  not  be  stopped  from  the  Aegon  dashboard.  In  order  to  input  the  feeds  directly  from  the 
camera  into  Aegon,  the  Axis  driver  had  to  be  loaded  on  the  (Microsoft  Windows  based)  Aegon 
server.  Configured  with  more  than  a  single  ground  camera  feed,  the  Aegon  process  would  slow 
down  considerably;  this  was  noticeable  during  testing  by  the  end  users  in  the  form  of  lag  (the 
amount  of  time  it  would  take  between  attempting  to  slew  the  camera  and  when  the  camera 
actually  slewed). 

Observation:  More  testing  needs  to  be  done  to  document  the  proper  settings  and 
configuration  necessary  for  Aegon  to  handle  the  processed  video  feeds  from  DAVION 
without  crashing  or  slowing  down. 

Aegon  Server 

PGSS  brought  both  a  primary  and  a  backup  Aegon  server  for  the  demo,  and  both  were  needed.  In 
addition  to  the  troubles  outlined  above,  the  primary  server  also  went  down  during  the  integration 
process  and  would  not  boot  back  up.  It  was  determined  that  a  bad  ribbon  cable  inside  the  server 
was  the  issue.  Since  the  ribbon  cables  only  purpose  is  to  connect  an  internal  capture  card  that 
wasn’t  being  used  for  this  exercise,  both  items  were  removed  and  the  server  then  booted. 

Observation:  One  of  the  lessons  learned  was  that  the  Aegon  representative  should  be  on  site 
for  the  entire  integration  process.  The  Aegon  representative  departed  after  the  initial 
integration  period  and  multiple  problems,  as  described  above,  occurred.  This  resulted  in 
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other  participants  being  pulled  away  from  their  own  technologies  to  trouble  shoot  the  Aegon 
issues.  It  is  likely  this  wasted  valuable  training  and  integration  time  as  the  Aegon 
representative  would  have  been  able  to  address  the  issues  more  efficiently. 


DAVION  Server  Setup 

The  Singapore  server  setup  included  5  Dell  Alienware  laptops: 

•  Video  Aggregator  :  One  laptop  collected  the  video  feeds  from  all  seven  cameras 

•  Video  Analysis:  Two  laptops  ran  video  analysis  on  feeds  from  the  aggregator 

•  Database  Server:  One  laptop  was  used  for  the  MySQL  database  server 

•  Workstation:  One  laptop  served  as  the  main  workstation  for  configuration 

The  Video  Aggregator  was  located  inside  of  the  GCS,  while  the  other  four  laptops  were  located 
on  a  table  next  to  the  GCS  and  connected  via  a  wired  Asus  gigabit  router. 

While  doing  a  dry  run  on  the  vignettes,  the  video  analytics  application  on  the  DAVION  server 
kept  crashing.  The  Singaporeans  were  able  to  make  code  changes  on  the  fly  to  fix  the  problems 
as  they  kept  popping  up. 

Observation:  More  work  is  needed  to  ensure  DAVION  is  a  fully  functioning  application 
and  service 

Mech  Assault  and  Tactical  Fiber 

For  network  connectivity  to  the  PDL  Combat  Town,  a  tactical  fiber  was  run  from  the  GCS  to  a 
building  inside  PDL  Combat  Town,  where  a  network  switch,  VOIP  phone,  and  Cisco  Wireless 
AP  were  placed.  During  the  Mech  Assault  this  tactical  fiber  was  run  over  by  one  of  the  AAV’s 
tracks  and  was  severed.  As  a  result  one  would  expect  that  all  three  cameras  in  the  PDL  Combat 
Town  would  drop  offline,  but  only  the  feed  from  Camera  6  was  lost;  at  the  time  it  was  the  only 
hardwired  camera,  while  Cameras  5  and  7  were  wireless;  the  mesh  network  configured  on  the 
Cisco  Access  Points  enabled  them  to  fall  back  to  using  wireless  once  the  wired  connection  was 
lost.  A  new  tactical  fiber  was  run  prior  to  the  main  demonstration  on  December  19th. 

Original  LTE  Plan 

During  the  final  site  survey  SMEs  from  JHU-APL  brought  equipment  to  conduct  initial  testing  of 
the  LTE  cell  phone  capabilities  at  PDL  Combat  Town.  In  conjunction  with  the  Pendleton  RF 
team,  it  was  determined  that  AT&T  is  using  Band  17  to  provide  commercial  cell  phone  service 
to  the  area.  Band  17  ranges  from  734  to  746  MHz  for  the  downlink;  AT&T’s  service  is  focused 
on  the  low  end  of  that  range.  This  was  also  the  same  range  the  PGSS  equipment  planned  to 
utilize  for  OUE  14.2.  To  prevent  overstepping  AT&T’s  commercial  service,  the  Pendleton  RF 
team  asked  PGSS  to  use  the  top  end  of  that  range  and  to  use  a  5  MHz  channel  instead  of  the 
usual  10  MHz.  Therefore  the  most  likely  range  used  by  the  Aerostat  for  LTE  service  for  OUE 
14.2  was  from  741  to  746  MHz.  It  was  noted  that  using  a  smaller  channel  would  result  in  lower 
bandwidth  available  to  the  phones.  However,  as  long  as  the  number  of  phones  was  limited  (less 
than  15  phones),  it  should  not  have  a  significant  effect  on  the  event.  The  Aerostat  could  also 
provide  a  wireless  N  network  to  the  smartphones,  which  was  the  backup  solution  if  there  were 
problems  utilizing  LTE  due  to  the  above  restrictions  or  other  interference. 


OUE  14.2  Final  Report 


39 


Final  LTE  Plan 

Ultimately  Band  4  was  used  instead  of  Band  17.  Band  4  uses  a  different  frequency  range  (1710  — 
1755  MHz  for  transmit,  21 10  -  2155  MHz  for  receive)  and  thus  required  a  different  Xiphos 
Radio  Unit  that  JHU-APL  brought  to  the  event.  This  band  resulted  in  less  interference  than  Band 
17  would  have  caused  to  AT&T  customers  in  the  area.  The  same  5  MHz  channel  size  restriction 
discussed  above  still  applied.  In  the  end  this  did  not  negatively  affect  the  test  due  to  the  limited 
number  of  smartphones  being  used.  While  the  Xiphos  Radio  unit  located  on  the  aerostat  is 
capable  of  outputting  60  watts,  most  of  the  exercise  was  run  at  10W.  Additional  wattage  does  not 
provide  additional  range  but  a  stronger  signal  within  the  same  range.  During  testing  it  was 
determined  that  outputting  10  watts  would  provide  adequate  coverage  for  the  exercise  area 
(Figure  22). 


Figure  22:  JHU-APL  and  PGSS  Technologists  Work  to  Integrated  the  4G  LTE  Capability 
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Note:  Singapore  Equipment  shaded  in  blue.  Orange  equipment  owned  by  JH-APL,  all  other  equipment  is  part  of  PGSS 

Figure  23:  Final  OUE  14.2  Network  Diagram 
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4G  LTE  Range  Testing 

This  section  provides  details  on  the  limited  range  testing  conducted  on  16  December  2014  as  part 
of  the  OUE  14.2  integration  and  demonstration  event.  Participating  JHU-APL  and  PGSS  team 
SMEs  conducted  basic  range  testing  on  the  PGSS  Mobile  Client  handhelds  using  the  4G  LTE 
capability.  Each  test  was  conducted  with  the  following  limitations: 

Limitations:  To  minimize  commercial  LTE  interference,  this  test  was  conducted  using  a 
band  4  radio  (instead  of  the  previously  tested  band  17  system)  with  transmit  power  decreased 
to  10W  (instead  of  60W)  and  a  patch  antenna  pointed  downward  (instead  of  an 
omnidirectional  antenna). 

Test  1 

Situation:  Weather  was  overcast,  but  otherwise  good.  Driver  with  a  single  device  started 
driving  from  the  PGSS  site  near  PDL  town  off  the  range  until  signal  loss.  The  PGSS  was 
used  to  measure  the  distance  between  the  start  location  and  each  stop  or  ending  location. 

Technical  Specifications 

UL  1732.5  MHz 
DL  2132.5 
Band  4 
Power  10W 

Findings:  At  1.27  miles  the  camera  feed  was  lost,  but  the  blue-force  tracking  icon  for  the 
driver  remained  and  the  Chat  feature  still  functioned.  The  driver  was  also  unable  to  make  a 
phone  call  at  this  distance.  When  the  driver  started  to  return  the  blue-force  icon  remained 
frozen  on  the  handheld  at  the  1.27  mile  location.  The  icon  for  the  driver  did  not  show  up  on 
the  GCS  at  this  distance.  At  .76  miles  the  driver  reappeared  on  the  GCS  and  on  the  driver’s 
handheld.  The  driver  was  able  to  slew  the  camera  at  .76  miles. 

Test  2 

Situation:  It  was  suggested  that  the  metal  body  of  the  vehicle  might  be  creating  some 
interference  so  during  this  run  the  driver,  and  an  additional  passenger  with  a  handheld, 
stepped  out  of  the  car  to  conduct  testing. 

Technical  Specifications 

UL  1732.5  MHz 
DL  2132.5 
Band  4 
Power  10W 
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Findings:  The  users  were  able  to  slew  the  camera,  Chat,  and  use  the  video  feed  at  .87  miles. 
The  levels  fluctuated  a  lot  at  this  range  and  the  signal  was  not  picked  up  inside  of  the  vehicle. 
At  1.01  miles  a  test  phone  call  was  completed  successfully  and  the  Chat  function  was  used. 
At  about  1.27  miles  the  vehicle  devices  dripped.  The  icons  disappeared  from  the  GCS  feed 
but  remained  on  the  handheld  device  inside  the  GCS.  The  users  rebooted  and  their  location 
reappeared  on  the  GCS.  On  the  return,  at  1.15  miles  the  users  were  able  to  successfully 
complete  a  phone  call  and  use  the  Chat  feature. 

Test  3 

Situation:  The  users  started  from  their  last  position  1.15  miles  with  power  raised  to  20W. 
All  phones  had  to  reconnect  before  proceeding. 

Technical  Specifications 

UL  1732.5  MHz 
DL  2132.5 
Band  5 
Power  20W 

Findings:  At  1.43  miles  the  users  were  able  to  successfully  complete  a  phone  call,  blue- 
force  tracking  worked,  and  they  were  able  to  successfully  use  the  Chat  feature. 

Conclusions:  Based  on  this  limited  testing,  the  aerostat-based  4G-LTE  system  performed  as 
expected  and  demonstrated  the  ability  to  support  the  mission,  providing  voice  (private  and  public 
network),  chat,  and  the  platform  video  and  common  operational  picture  to  the  warfighter.  For 
this  exercise,  the  LTE  range  was  intentionally  restricted  to  cover  only  the  exercise  area  of 
operations;  previous  testing  of  the  system  demonstrated  these  mission  functions  out  to  10  miles. 


OUE  Technology  Integration  Functional  Areas,  Objectives,  and 
Measures 

This  section  identifies  the  measures  that  were  used  to  collect  data  on  the  integration  of  the  PGSS, 

VA,  and  DAVION  capabilities  during  OUE  14.2.  The  TEC  assessment  team  addressed  the  measures 
using  the  data  sources  identified  in  the  following  tables.  All  results  are  supported  by  observations  and 
user  comments  when  available. 

Functional  Area  A-l:  Integration  and  Interoperability 

Can  the  PGSS,  PGSS  Mobile  COP,  VA,  and  the  DAVION  applications  successfully  work 
together  to  support  scenario  requirements? 

Objective  A-l.l:  Technology  Integration 

This  objectives  seeks  to  observe  and  assess  the  integration  of  the  identified  technologies.  In  this 
context  integration  is  defined  as  the  merging  or  combining  of  two  or  more  components  or 
configuration  items  into  a  higher  level  system  element,  and  ensuring  that  the  logical  and  physical 
interfaces  are  satisfied  and  the  integrated  system  satisfies  its  intended  purpose. 
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Table  16:  Objective  A-l.l  Data  Matrix 


Measure 

Source 

Product 

ObjectiveA-1.1:  Network/Camera  Integration 

Measure  A-l-1-1:  Steps  required  to  integrate  the  PGSS, 
PGSS  mobile  COP,  VA,  and  DAVION 

Event  Logs,  Interview, 
Questionnaire 

Table,  Text 

Measure  A- 1-1-2:  Issues  arising  from  the  integration  of 
PGSS,  PGSS  mobile  COP,  VA,  and  DAVION 

Event  Logs,  Interview, 
Questionnaire 

Text 

Measure  A- 1-1-3:  SME  Rating  of  the  integration  of  the 
identified  technology 

Interview,  Questionnaire 

Text,  Bar  Chart 

Objective  A-1.2:  Technology  Interoperability  and  Dissemination 

This  objectives  seeks  to  observe  and  assess  the  interoperability  of  the  identified  technologies,  and  the 
ability  of  those  technologies  to  effectively  disseminate  needed  information.  In  this  context 
interoperability  is  defined  as  the  ability  of  two  or  more  systems  or  components  to  exchange 
information  and  use  the  information  that  has  been  exchanged.  In  this  context  effective 
dissemination  is  the  ability  for  the  identified  technology  to  provide/share  the  data  specific  to  that 
technology,  as  expected,  using  the  network  configuration  designed  to  support  the  event.  Current 
excepted  range  of  the  PGSS  is  defined  as  10  miles. 

Additionally,  event  logs  will  be  used  to  record  any  issues,  to  include  the  circumstances,  impact  on  the 
mission,  technology  configuration  changes,  and  resolutions. 


Table  17:  Objective  A-1.2  Data  Matrix 


Measure 

Source 

Product 

Objective  A-1.2:  Technology  Interoperability  and  Dissemination 

Measure  A- 1-2-1:  Observations  on  the  overall 
interoperability  of  the  identified  technologies 

Interview,  Questionnaire 
Data  collectors  Observations, 

Table,  Text 

Measure  A- 1-2-2:  Interoperability  issues  with  the 
identified  technologies 

Event  Logs 

Interview,  Questionnaire 

Table,  Text 

Measure  A- 1-2-3:  User  Ratings  of  the  successful 
interoperability  of  the  identified  technologies 

Interview,  Questionnaire 

Text,  Bar  Chart 

Measure  A- 1-2-4:  The  identified  technology 
disseminated  data  as  expected 

Interview,  Questionnaire 
Data  collectors  Observations, 

Table,  Text 

Measure  A- 1-2-5:  Determine  the  effective  range  of  the 
identified  technology 

Event  Logs 

Interview,  Questionnaire 

Table,  Text 

Measure  A-l-2-6:  Validate  the  PGSS  telephone 
capability 

Event  Logs 

Interview,  Questionnaire 

Table,  Text 

OUE  Integration  Survey  Results 

This  section  provides  the  survey  feedback  collected  from  participating  users  and  SMEs  during  the 
OUE  14.2  technology  integration  event. 

Measure  A-l-1-1:  Steps  required  to  integrate  the  PGSS,  PGSS  mobile  COP,  VA,  and 
DAVION 

•  This  measure  is  addressed  in  detail  in  the  previous  section  “Technology  Integration” 
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Measure  A-l-1-2:  Issues  arising  from  the  integration  of  PGSS,  PGSS  mobile  COP,  VA,  and 
DAVION 

•  This  measure  is  addressed  in  detail  in  the  previous  section  “Technology  Integration” 

Measure  A-l-1-3:  SME  Rating  the  integration  of  the  identified  technology 
The  process  to  integrate  the  PGSS  with  other  technologies  during  OUE  14.2  was  successful. 


Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 

The  process  to  integrate  the  PGSS  Mobile  COP  with  other  technologies  during  OUE  14.2 

was  successful. 


Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 
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The  process  to  integrate  the  VA  with  other  technologies  during  OUE  14.2  was  successful. 


B 

Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 

The  process  to  integrate  DAVION  applications  with  other  technologies  during  OUE  14.2 

was  successful. 


Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


If  you  chose  an  answer  other  than  Largely  or  Completely  Agree  in  one  of  the  above  questions 
please  describe  any  integration  issues  you  recall  with  PGSS,  PGSS  Mobile  COP,  VA,  and/or 

the  DAVION  applications,  and  provide  feedback  on  the  integration  process  as  a  whole. 

•  One  of  the  things  that  would  have  helped  the  integration  process  would  be  to  know  more 
about  the  infrastructure  on  the  terrain  to  setup  a  better  wireless  network  for  devices  to 
communicate  on  the  network.  There  was  a  lot  of  interference  that  caused  lag  on  the 
network  due  to  buildings.  Also  to  learn  more  about  the  different  technologies  involved 
prior  to  deployment. 

•  Network,  power,  camera  issues  were  encountered  on  a  daily  basis 

•  We  managed  to  get  the  LTE  capabilities  working  fairly  easily.  I  largely  attribute  this  to  a 
significant  amount  of  preparation  testing  in  the  lab  and  in  the  field 

•  For  integration  between  PGSS  Mobile  COP  and  VA,  perhaps  a  more  advanced  technical 
information  exchange  should  have  been  done.  I  got  the  feeling  that  software  changes 
were  being  done,  which  could  have  been  done  prior  to  this  test 
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•  The  DAVION  application  continued  to  crash  the  COP  servers,  this  I  believe  could  be 
resolved  by  DRG  and  the  DAVION  reps  communicating  to  resolve  this  issue.  The  COP 
does  not  run  like  a  normal  operating  system,  therefore  research  between  the  two  would 
greatly  benefit  this  effort.  As  a  whole  the  integration  process  went  well,  the  extra  time 
allocated  to  this  process  help  greatly. 

Measure  A-l-2-1:  Observations  on  the  overall  interoperability  of  the  identified  technologies 
Measure  A-l-2-2:  Interoperability  issues  with  the  identified  technologies 

What  would  you  describe  as  the  biggest  challenge  to  integrating  the  technologies  that 

participated  in  OUE  VM14.2?  (SMEs) 

•  Change  in  schedule  and  weather 

•  Availability  of  the  guards 

•  *Guards  visited  two  days  earlier  than  scheduled  significantly  reducing  integration 
time 

•  Stabilizing  the  network  for  Wi-Fi  and  4G 

•  Time  changes  for  setting  up  the  stations 

•  Communication  and  synchronization  between  different  parties  working  on  the  different 
technologies 

•  One  of  the  biggest  challenges  was  integrating  some  of  the  unknown  technologies  for  the 
first  time  without  going  through  an  R&D  phase.  It  would  have  been  easier  to  test  the 
different  technologies  together  prior. 

•  The  integration  of  the  DAVION  software  in  to  the  Ageon  software.  It  was  initially 
difficult  to  get  it  to  work  but  once  the  DAVION  software  was  routed  straight  into  Ageon 
it  seemed  to  work  well.  It  was  a  matter  of  having  more  time  to  integrate  and  test  the 
software  prior  to  the  actual  mission.  If  we  were  allowed  the  initial  two  weeks  to  test  and 
integrate  all  issues  would  have  been  worked  out  prior  to  the  actually  week  of  testing.  I 
believe  the  fix  would  be  to  allow  for  longer  integration  and  testing  to  “BURN”  in  the 
software. 

•  The  largest  challenge  was  integrating  the  COP  servers  with  the  DAVION  software. 

•  The  amount  of  time  that  we  had  to  integrate  the  systems.  Most  of  the  technologies  had  to 
be  set  up  on  the  spot  and  needed  to  be  up  and  running  within  3  days 

•  *Guards  visited  two  days  earlier  than  scheduled  significantly  reducing  integration 
time 

•  Fiber  optics 

•  The  biggest  challenge  was  contention  for  the  resources.  Everyone  was  trying  to  use  the 
same  cameras  and  at  times  our  goals  conflicted.  In  the  end  it  all  worked  well. 

•  From  an  LTE  payload  perspective,  frequency  clearance  was  a  major  challenge.  The 
payload  was  developed  to  operate  in  areas  where  LTE  Band  17  (700  MHz- AT&T  owned 
in  most  areas  of  CONUS)  is  not  used.  In  order  to  operate  at  Camp  Pendleton,  we 
(JHU/APL)  had  to  make  a  last  minute  switch  to  the  LTE  Radio  &  antenna  setup.  APL 
worked  with  NAVAIR  to  resolve  the  issue,  but  additional  consideration  should  be  given 
to  frequency  clearance  in  the  future. 

•  We  needed  PGSS  at  a  fixed  position.  This  caused  some  integration  difficulties  with  VA 

•  Network  drops 
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•  The  biggest  challenge  was  integrating  systems  into  the  GCS  that  have  never  been 
integrated  before  but  thankfully  all  the  right  personnel  were  on  site  to  complete  the  task 
in  a  timely  manner.  Some  of  the  necessary  equipment  wasn’t  brought  out  on  the  first  day 
such  as  CAT5  cable  but  luckily  most  of  the  equipment  was  easily  obtainable  at  local 
stores. 

•  The  integration  time  was  too  short 

•  The  time  was  too  short  for  integrating  VA  given  the  unfamiliar  environment.  The 
environment  posed  some  unexpected  challenges  for  the  ad-hoc  camera  and  data  network 
setup,  which  in  the  end  delayed  the  progress  of  the  VA  software 

•  Hardware  failures;  network,  generator,  cameras,  etc. 

•  Early  provision  of  VA  streams  to  PGSS  might  have  potentially  avoided  some  issues 
regarding  stream  playback  on  PGSS  COP 

•  Schedule  cuts;  reduction  in  the  number  of  days  for  integration/testing  etc.,  shortened  the 
buffer  time  to  cater  for  unforeseen  hiccups. 

•  Network  Issues 

•  Limited  cooperation  between  teams 

•  Misunderstandings  about  technology  requirements 

•  Schedule  changes 

•  The  final  requirements  not  conforming  to  the  original  expectations 

What  would  you  describe  as  the  biggest  success  during  the  OUE  14.2  integration  process? 

(SMEs) 

•  All  technologies  are  working  together 

•  Full  integration  of  PGSS/V  A/D  AVION  as  unique  systems;  PGSS  as  sensor,  VA  for  the 
analytics  and  DAVION  as  user  interface 

•  Integration  with  VA  and  DAVION  handsets  was  largely  successful  as  compared  to  XVM 
14.1 

•  One  of  the  biggest  successes  was  that  we  achieved  full  system  functionality  with  the 
limited  time  we  had  to  test 

•  The  biggest  success  was  the  full  integration  of  all  the  systems  being  presented  before  the 
demonstration  actually  took  place. 

•  Integration  of  PGSS  video  feed  into  VA  processing 

•  Incorporation  of  all  sensors  and  VA  processing  machines  onto  a  single  combined  network 

•  Vignette  demos  that  incorporate  PGSS,  DAVION,  and  VA  are  successful 

•  The  promise  of  the  conceived  system  which  includes  PGSS,  DAVION,  and  VA  is 
enormous,  if  the  integration  can  be  seamless.  The  OUE  14.2  exercise  is  significant  to 
demonstrate  the  mentioned  potential. 
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•  Logistics  was  a  success!  The  logistics  went  well.  All  pieces  of  PGSS  gear  arrived  on 
trucks  and  were  set  in  place  by  crane.  The  Inflation  and  aerostat  set  up  went  quickly 
which  allowed  for  more  time  in  setting  up  and  testing  the  Common  Operating  Picture 
inside  the  GCS  and  on  the  mobile  handheld  devices.  The  Logistics  team  also  did  an 
outstanding  job  supplying  the  team  with  any  equipment  that  was  needed  to  include  fuel 
for  the  two  generators.  Having  fuel  brought  in  daily  was  crucial  to  the  success  of  the 
PGSS  mission.  The  final  day  went  as  smooth  as  the  inflation  day.  The  aerostat  was 
deflated  in  record  time  and  the  conexes  were  packed  and  ready  for  the  trucks  and  crane  to 
arrive  and  load  all  the  equipment  for  transport.  All  the  generators  had  the  fuel  ran  off  and 
ready  for  movement. 

•  Vignette  3,  we  can  stream  video  to  DAVION  that  is  was  lacking  in  XVM  14.1.  Vignette 
1  was  also  successful. 

•  The  Oceus  network  seem  to  work  seamlessly.  As  the  WIFI  network  seemed  to  struggle, 
when  the  Oceus  network  had  no  distance  or  lagging  issues. 

•  Deployment  of  the  LTE  payload  and  interoperability  with  the  Singaporean  applications 
was  a  major  success  from  my  perspective.  Working  together  facilitated  intercultural 
interaction. 

•  I  would  describe  the  soldier  training  as  a  big  success.  They  seemed  to  grasp  things  very 
well.  I  think  this  showed  our  applications  were  fairly  intuitive.  I  also  got  a  lot  of  great 
feedback. 

•  Managing  to  showcase  the  applications  during  the  mech  assault 

•  DAVION  applications  were  able  to  integrate  the  VA  and  PGSS  video  streams 

•  Integrating  all  of  the  technologies  successfully 

•  Measure  A- 1-2-2  is  also  addressed  in  the  previous  section  “Technology  Integration” 

SME  Lessons  Learned 

Are  there  any  lessons  learned  you  will  take  away  from  this  event  regarding  your  technology, 
someone  else’s  technology,  or  about  the  integration  process? 

•  The  biggest  lesson  would  be  to  take  the  time  to  learn  and  fully  understand  all 
technologies  involved  within  the  project  to  better  integrate  them  into  one  system.  I  feel 
the  demo  went  well  and  we  worked  through  each  problem  effectively. 

•  There  was  configuration  needed  to  use  the  LTE  with  the  smart  phones  that  shouldn’t  be. 
The  configuration  required  *text  not  readable*  software. 

•  Some  technical  aspects  of  the  different  technologies  could  have  been  discussed  and 
agreed  on  beforehand  to  make  the  integration  process  as  smooth  as  possible.  Ex.  The 
different  URL  video  streams,  the  cameras’  IP  addresses 

•  Different  people  will  have  different  ways  of  building  their  applications 

•  PGSS  is  quite  interesting.  I  have  an  ideal  to  use  PGSS  to  scan  the  area  and  automatically 
detect  objects 

•  I  think  the  lesson  is  that  more  TIME  is  needed  when  integrating  foreign  sensors  and 
software  into  the  already  existing  PGSS  software  suite.  The  more  time  allowed  to  test  and 
evaluate  the  new  technology  the  better.  All  bugs  can  be  worked  out  and  the  system  can  be 
pushed  to  its  limits  prior  to  the  actual  event.  There  is  nothing  worse  than  trying  to  train 
people  on  software  when  it  is  not  integrated  properly  Or  hasn’t  been  pushed  to  its  limits 
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prior  to  handing  the  mobile  devices.  I’ve  been  on  several  of  these  missions  and  they  are 
always  cut  short.  I  believe  more  time  on  the  front  end  of  the  mission  will  pay  dividends 
on  the  back  end  during  the  actual  demonstration.  Thank  you  for  the  opportunity  to  give 
my  input. 

•  Situational  awareness  is  of  the  upmost  importance  for  many  military  situations. 
Situational  awareness  through  technology  is  an  effective  means,  as  long  as  the 
technology  becomes  robust,  user-friendly,  and  accessible 

•  Streamline  the  workflows 

•  To  communicate  with  other  parties  prior  to  test  execution  to  resolve  any  outstanding 
issues. 

•  We  learned  to  use  android  applications  to  ping  the  cameras,  that  improved  camera  setup 
efficiency 

•  More  time  for  integration 

•  I  think  the  vendors  should  have  discussed  their  needs  to  other  vendors  prior  to  arrival  so 
integration  could  run  as  smooth  as  possible. 

•  Mitigate  schedule  changes 

•  Minimize  vignette  amendments 

•  PGSS  camera  provides  really  good  imagery 

•  Improve  robustness  to  special  conditions 

•  New  tools  to  improve  efficiency 

•  Got  to  have  backup  plans 

•  Bring  all  of  the  test  equipment  you  have,  even  if  you  don’t  think  you  will  need  it 

•  This  was  not  my  first  integration  event  so  I  am  not  surprised  by  much,  nothing  seemed 
out  of  the  ordinary.  I  did  get  great  feedback  on  potential  improvements 

•  Frequency  clearance  should  be  addressed  earlier  in  the  planning  phase  for  the  PGSS  LTE 
payload 


Measure  A-l-2-3:  User  Ratings  the  successful  interoperability  of  the  identified  technologies 
Measure  A-l-2-4:  The  identified  technology  disseminated  data  as  expected 


The  D AVION  Map  application  successfully  disseminated,  exchanged,  and  used  information 

with  the  other  OUE  14.2  technologies.  (Users) 


7 

6 

5 

4 

3 

2 

1 

0 


Completely  Largely 
Disagree  Disagree 


Somewhat  Somewhat  Largely 
Disagree  Agree  Agree 


Completely  Not 
Agree  Applicable 
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The  D AVION  Augmented  Reality  application  successfully  disseminated,  exchanged,  and  used 
information  with  the  other  OUE  14.2  technologies.  (Users) 


6 


Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


The  D AVION  Chat  application  successfully  disseminated,  exchanged,  and  used  information 

with  the  other  OUE  14.2  technologies.  (Users) 


Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


The  D AVION  WhiteBoarding  application  successfully  disseminated,  exchanged,  and  used 
information  with  the  other  OUE  14.2  technologies.  (Users) 

6 
5 
4 
3 
2 
1 
0 

Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


If  any  technologies  were  not  ranked  as  Largely  Agree  or  Completely  Agree  please  provide 

comments  to  clarify  your  ranking  (Users): 

•  The  location  of  people  and  places  was  not  very  accurate 

•  Map  did  not  update,  not  very  accurate 

•  Augmented  Reality  worked  but  not  very  accurate 
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•  The  Augmented  Reality  should  change  the  distance  when  we  get  closer  so  that  we  would 
know  our  location  wasn’t  in  the  same  spot  as  the  pin-point  pointed 

•  The  Augmented  Reality  application  is  not  really  accurate.  It  is  out  of  point  of  where 
things  are  pinned  and  requires  adjusting  to  the  location  where  we  put  it. 

•  The  Augmented  Reality  wasn’t  on  point  and  wasn’t  as  accurate  as  it  should  be 

Please  provide  feedback  on  the  overall  Interoperability  of  the  OUE  14.2  technologies: 

•  Overall  good  if  the  three  aps  (Maps,  Chat,  and  Whiteboarding)  could  be  integrated  into 
one  app.  It  would  save  time  not  having  to  switch  through  apps. 

•  It  was  good  except  for  the  WiFi  capabilities 

•  Overall  it  was  good  and  just  needs  a  little  improvement 

•  It  was  good  and  easy  to  use.  Just  needs  some  improvement 

•  In  theory,  effective.  In  application,  not  very  probable.  WiFi  capabilities  are  in  and  out, 
the  apps  are  not  interconnected  so  you  constantly  have  to  exit  and  re-enter  apps,  harder  to 
share  information  quickly.  Scenarios  we  practiced  were  scripted. 

•  Worked  well  when  WiFi  signal  was  strong.  Lost  signal  a  lot. 

•  The  system  was  smooth 

•  Changing  from  app  to  another  was  easy  and  simple 

Measure  A-l-2-5:  Determine  the  effective  range  of  the  identified  technology 

•  Please  see  the  4G  LTE  Range  Testing  section 

Measure  A-l-2-6:  Validate  the  PGSS  telephone  capability 

•  The  PGSS  telephone  capability  was  validated  during  multiple  vignettes.  Calls  were 
placed  between  networked  phones,  and  to  and  from  phones  outside  of  the  network. 
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DAVION  AND  PGSS  USER  TRAINING  AND  VIGNETTES 

In  order  to  provide  Singaporean  Guard  and  the  U.S.  Marine  users  exposure  to  the  DAVION 
applications  and  PGSS  Mobile  Client  the  OUE  team  conducted  training  events  for  each  group 
(Figures  24  and  25).  Prior  to  training,  each  group  was  asked  to  complete  a  demographics  form. 
The  Singaporean  Guards  participated  in  training  and  practice  vignettes  on  11  December  2014. 
The  users  were  divided  into  two  groups,  2  DAVION  and  3  PGSS  users.  The  U.S.  Marines  were 
trained  on  14  December  2014.  10  Marines  were  sent  for  training,  but  only  5  remained  to 
participate  in  the  practice  vignettes  and  data  collection  efforts.  The  Marines  were  divided  into 
two  groups  of  5,  one  for  DAVION  and  one  for  PGSS.  Each  group  was  provided  a  verbal  and 
visual  demonstration  of  their  respective  technology  by  SMEs.  During  training  users  were 
encouraged  to  explore  the  capabilities  of  the  technologies  and  ask  any  pertinent  questions. 


Figure  24:  Singaporean  Guards  Receive  Training  on  DAVION  (Left)  and  PGSS  (Right) 


Figure  25:  U.S.  Marines  Receive  Training  on  DAVION  (Left)  and  PGSS  (Right) 


Vignette  1 

After  training  was  completed  users  worked  through  a  series  of  vignettes  to  help  them  gain 
exposure  and  familiarity  with  the  technologies.  Vignette  1  focused  on  a  patrol  scenario  and 
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utilized  the  VA  technology  to  alert  users  when  the  patrol  deviated  from  the  expected  route. 
Vignette  1  was  run  with  the  DAVION  applications  and  the  PGSS  Mobile  Client  separately  with 
each  user  acting  in  each  of  the  designated  roles  for  the  scenario  (Figure  26).  During  the  vignettes 
the  users  were  able  to  provide  feedback  on  the  vignette  so  that  adjustments  could  be  made,  when 
possible,  to  help  add  realism  to  the  scenario.  Due  to  the  Singaporean  Guard  arriving  two  days 
early  for  training,  the  VA  system  was  not  yet  functioning  and  the  alerts  were  simulated. 


Figure  26:  Patrol  Users  Approach  and  Tag  Suspicious  Items  as  Part  of  Vignette  1  (Left) 
and  Acting  HQ/V CP  Users  Receive  Intel  on  Suspicious  Items 


Vignette  1  OUE  Data  Collectors’  Observations  and  User  Feedback 

As  part  of  the  data  collection  effort  the  data  collectors  recorded  technology  issues,  changes  to 
vignette  1,  and  user  feedback  during  many  of  the  training  vignette  runs.  The  data  collected 
during  each  of  those  runs  is  provided  below. 

Vignette  1, 1st  Run,  PGSS  (Singaporean  Guards) 

•  Vignette  was  altered  to  include  the  GCS  Chatting  with  the  user  and  a  user  request  for  the 
GCS  to  zoom  the  camera  after  slewing 

•  Text  feature  did  not  work 

•  No  video  feeds  were  available  (issue  likely  because  of  too  many  connections) 

•  Vignette  was  not  completed  because  of  a  system  failure 

•  Feedback:  Old  chats  should  be  cleared  before  devices  are  given  to  new  users.  Users 
should  have  programed  names  instead  of  numbers  that  make  it  hard  to  identify  the  user 

Vignette  1, 1st  Run,  PGSS  (U.S.  Marines) 

•  Aegon  was  not  working  on  the  devices,  but  was  fixed  once  the  power  was  cycled 

•  It  took  two  full  minutes  before  the  tag  showed  up  on  the  VCP  device 

•  Feedback:  The  VCP  user  liked  the  capacity  for  this  technology  to  help  with  recovery. 
The  user  does  not  receive  an  alert  when  there  is  an  uploaded  picture  so  unless  they  know 
something  is  coming  they  do  not  know  to  look  for  the  information. 
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Vignette  1,  2nd  Run,  PGSS  (Singaporean  Guards) 

•  The  VCP  users  added  additional  communication  with  the  GCS  for  their  situational 
awareness  during  the  vignette 

•  Users  were  asked  to  use  the  Chat  All  feature  so  that  all  users  had  situational  awareness 

•  Feedback:  The  Map  has  a  limitation  on  zooming  so  pinning  locations  is  not  completely 
accurate.  Switching  between  views  was  a  bit  annoying.  Having  all  of  the  information 
in  one  view  would  be  good. 

Vignette  1,  2nd  Run,  PGSS  (U.S.  Marines) 

•  Had  to  make  a  2nd  attempt  to  slew  the  camera 

•  Server  dropped.  Vignettes  were  halted  until  the  issue  was  resolved. 

Vignette  1,  3rd  Run,  PGSS  (Singaporean  Guards) 

•  In  spite  of  the  accuracy  of  the  pinning  being  slightly  off  due  to  zooming  limitations,  the 
VCP  user  was  able  to  navigate  to  within  eyesight  of  the  patrol  users  using  the  recovery 
pin  on  the  Map. 

•  The  user  attempted  twice  to  slew  the  camera  without  success.  The  issue  was  identified  as 
a  problem  caused  due  to  the  attempt  to  integrate  the  VA  cameras  while  vignettes  were 
being  conducted.  The  PGSS  camera  was  rebooted  and  the  issue  was  resolved. 

•  Feedback:  The  screens  are  difficult  to  see  in  certain  light  due  to  the  glare. 

Vignette  1,  3rd  Run,  PGSS  (U.S.  Marines) 

•  Error  sending  tag/waypoint,  the  waypoint  failed  to  save 

Vignette  1, 1st  Run,  DAVION  (Singaporean  Guards) 

•  Blue  Force  tracking  is  slow  to  update 

•  User  sent  a  pic  using  the  Chat  capability  and  tagged  it  on  the  Map 

•  The  Blue  Force  tracking  updated  so  slowly  it  could  not  be  used  to  navigate 

•  Feedback:  The  WhiteBoarding  app  does  not  show  the  users  position  so  it  cannot  be  used 
to  help  navigate  when  making  plans.  The  WhiteBoarding  user  should  show  up  on  the 
app. 

Vignette  1, 1st  Run,  DAVION  (U.S.  Marines) 

•  VCP  user  notified  patrol  after  they  noted  their  recovery  area  in  WhiteBoarding  that  he 
had  changed  the  location 

•  The  Blue  Force  Tracking  appears  to  be  updating  more  quickly  than  during  Guard  training 

•  Vignette  stopped  due  to  loss  in  Wi-Fi  signal 

Vignette  1,  2nd  Run,  DAVION  (Singaporean  Guards) 

•  The  vignette  was  changed  to  have  users  use  the  Map  tagging  feature  instead  of 
WhiteBoarding  because  the  VCP  user  could  not  navigate  to  the  patrol  using  the 
WhiteBoarding  app  due  to  their  position  not  showing  up  on  the  whiteboard.  Both  patrol 
users  tagged  the  recovery  location  to  see  if  that  forced  a  refresh  of  the  slow  Blue  Force 
locations.  It  did  not  force  a  refresh,  one  user  still  appeared  on  the  other  side  of  the  street. 
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Vignette  1,  2nd  Run,  DAVION  (U.S.  Marines) 

•  All  features  worked  during  this  run 

•  Feedback:  All  of  the  apps  should  be  integrated  into  one  app  instead  of  the  user  having  to 
switch  between  them 

Vignette  1,  3rd  Run,  DAVION  (Singaporean  Guards) 

•  The  user  tagged  the  boxes,  but  because  his  GPS  location  was  not  updated  the  tagged 
position  was  not  correct. 

•  The  tags  for  the  recovery  location  did  not  appear. 

•  Users  sent  the  suspicious  items  information  through  Chat  and  Map  feature 

•  Feedback:  The  lag  makes  it  hard  to  determine  if  forces  are  Red  or  Blue.  The  blue  and 
purple  tag/pin  colors  are  too  close.  It  is  difficult  to  distinguish  between  the  two. 
Landmarks  and  Blue  Force  icons  should  not  be  so  close. 

Vignette  1,  3rd  Run,  DAVION  (U.S.  Marines) 

•  No  alert  received  after  route  deviation. 

•  Photo  sending  lagged/was  not  received 

•  Feedback:  Need  to  be  able  to  zoom  more  closely.  If  there  are  multiple  tags  within  a 
small  area  the  user  is  not  able  to  zoom  close  enough  to  click  on  the  tag  they  want.  The 
user  should  be  able  to  tap  on  the  alert  and  have  the  map  open  and  indicate  the  alert 
location. 

Vignette  2 

Vignette  2  focused  on  a  suspicious  vehicle  scenario  and  utilized  the  VA  technology  to  alert  users 
when  the  vehicle  had  a  change  in  number  of  pax,  and  also  when  the  passengers  engaged  in  an 
altercation  with  VCP  guards.  Vignette  2  was  run  with  the  DAVION  applications  and  the  PGSS 
Mobile  Client  separately  with  each  user  acting  in  each  of  the  designated  roles  for  the  scenario 
(Figure  27).  During  the  vignette  the  users  were  able  to  provide  feedback  on  the  vignette  so  that 
adjustments  could  be  made,  when  possible,  to  help  add  realism  to  the  scenario.  Due  to  the 
Singaporean  Guard  arriving  two  days  early  for  training  the  VA  system  was  not  yet  functioning 
so  the  alerts  were  simulated. 


Figure  27:  Patrol  User  Subdues  Vehicle  Passenger  during  Vignette  2  (Left)  Patrol  User 
Sends  Images  of  Suspects  (Center)  and  VCP  User  Receiving  Updates  (Right) 
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Vignette  2  OUE  Data  Collectors’  Observations  and  User  Feedback 

As  part  of  the  data  collection  effort  the  data  collectors  recorded  technology  issues,  changes  to 
vignette  2,  and  user  feedback  during  many  of  the  training  vignette  runs.  The  data  collected 
during  each  of  those  runs  is  provided  below. 

Vignette  2, 1st  Run,  PGSS  (Singaporean  Guards) 

•  Vignette  altered  to  include  the  GCS  sending  a  Chat  to  users  indicating  the  area  of  the 
suspicious  items 

•  The  Chat  lagged  and  photos  at  the  vehicle  location  could  not  be  sent 

•  The  VCP  user  did  not  see  Chats  sent  by  the  patrol  because  they  sent  to  “super,”  Chat  All 
should  be  used. 

•  Feedback:  Chat  should  have  names  to  help  identify  who  you  are  sending  to.  Chat 
should  have  a  copy  and  paste  feature. 

Vignette  2, 1st  Run,  PGSS  (U.S.  Marines) 

•  All  features  worked  successfully 

Vignette  2,  2nd  Run,  PGSS  (Singaporean  Guards) 

•  The  login  for  some  of  the  devices  was  not  working.  The  users  tried  turning  the  Wi-Fi  on 
and  off.  Ultimately  the  Aegon  SME  had  to  address  the  issue. 

•  The  pin  location  was  off  again  due  to  lack  of  zooming 

Vignette  2,  2nd  Run,  PGSS  (U.S.  Marines) 

•  All  features  worked  successfully 

Vignette  2, 1st  Run,  DAVION  (Singaporean  Guards) 

•  The  refresh  was  slow  making  it  difficult  to  relocate  the  pin 

•  WhiteBoarding  experienced  a  lag  causing  the  patrol  to  have  to  wait  for  updating 

•  The  VCP  user  lost  connection  and  was  logged  out  of  the  applications 

•  Feedback:  The  applications  force  the  user  to  take  a  photo  in  order  to  pin  something. 
Users  thought  this  should  not  be  required.  Users  suggested  that  a  button  be  created  that 
forces  the  applications  to  refresh. 

Vignette  2, 1st  Run,  DAVION  (U.S.  Marines) 

•  VA  alerts  not  operating 

•  Feedback:  If  something  is  tagged  the  users  should  get  a  ping  to  notify  them 

Vignette  2,  2nd  Run,  DAVION  (Singaporean  Guards) 

•  Pin  did  not  work,  all  other  actions  good 

Vignette  2,  2nd  Run,  DAVION  (U.S.  Marines) 

•  Wi-Fi  dropped  on  all  of  the  devices,  all  features  were  working  as  expected  before  the 
drop 
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Vignette  3 

The  focus  of  Vignette  3  was  to  provide  users  with  exposure  to  potential  uses  of  PGSS,  VA,  and 
DAVION  technologies  within  a  mechanized  assault  scenario.  Users  were  asked  to  work  through 
the  vignettes  on  their  own  to  try  to  gain  familiarity  with  the  applications  within  the  provided 
context.  The  users  were  provided  an  attacker  and  a  defender  scenario.  In  place  of  specific  data 
being  collected  during  Vignette  3  runs,  users  were  asked  to  provide  comments  on  their  use  of  the 
technologies  during  the  actual  mechanized  assault  as  part  of  the  final  user  survey. 

User  Comments: 

•  During  the  assault  on  PDL  Combat  Town  we  were  able  to  communicate  and  pass  info  about 
where  the  enemy  was,  mooring,  and  deploying  using  the  Chat  capabilities  and  camera  feeds. 
By  using  the  cameras  and  Chat  apps  we  were  able  to  employ  surprise  against  the  enemy 
because  we  didn’t  have  to  poke  out  heads  out  to  see  where  they  were,  and  give  away  our 
position.  However,  once  we  were  engaged,  due  to  the  fact  that  the  apps  were  not 
interconnected,  it  would  take  too  much  time  to  use  the  apps  effectively  while  engaging  the 
enemy.  If  the  apps  were  interconnected  we  could  have  continued  to  track  the  enemy 
movements  to  better  engage  them. 

•  For  the  attack  I  was  the  eyes  in  the  sky.  So  when  the  enemy  was  near  I  was  watching  on  the 
monitors  and  was  allowed  to  communicate  using  the  Chat  application.  I  let  my  team  know 
when  the  enemy  was  near  the  building,  how  they  were  moving,  and  how  many  were  moving 
up  to  the  building.  If  it’s  possible  to  get  a  greater  wireless  connection  it  would  make  this 
program  way  more  efficient  in  a  battle  area. 

DV  Day  Vignette 

The  DV  Day  vignette  was  rehearsed  on  the  previous  day  and  the  morning  of  the  actual 
demonstration.  The  DAVION  and  PGSS  teams  were  integrated  to  provide  visitors  with  a 
demonstration  that  incorporated  all  of  the  participating  technologies  (Figure  28).  To  help 
provide  users  with  an  engaging  demonstration  monitors  were  setup  in  the  demonstration  area  to 
provide  the  visitors  an  on  screen  display  of  the  PGSS  GCS  aerial  view,  the  HQ/VCP  Users 
DAVION  handset,  and  the  HQ/VCP  users  PGSS  handset.  No  additional  data  was  collected 
during  the  demonstration,  but  all  technologies  worked  as  expected  resulting  in  a  very  successful 
demonstration. 


Figure  28:  DV  Day,  HQ  Users  at  Mobile  COP  (Left)  Patrol  Team  Receiving  Orders 
(Center)  and  Combined  User  Team  Subduing  Suspects  during  DV  Day  Rehearsal  (Right) 
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DAVION  ASSESSMENT  RESULTS 


This  section  provides  details  on  the  functional  areas,  objectives,  and  measures  used  to  collect 
data  on  the  DAVION  suite  of  applications.  It  also  includes  the  survey  feedback  collected  during 
the  event. 

DAVION  Applications  Functional  Areas,  Objectives,  and  Measures 

This  section  identifies  the  measures  used  to  collect  data  on  the  suitability  of  the  DAVION 
Applications  during  OUE  14.2.  The  TEC  assessment  team  addressed  the  measures  using  the  data 
sources  identified  in  the  following  tables.  All  results  are  supported  by  observations  and  user 
comments  when  available. 

Functional  Area  B-l:  Suitability 

Are  the  DAVION  Applications  suitable  for  supporting  operations  in  urban  environments? 
Objective  B-l.l:  Assess  Usability 

This  objectives  seeks  to  assess  the  usability  of  the  identified  technologies.  In  this  context  usability  is 
includes  the  overall  look  and  feel,  video  quality,  and  timeliness  of  data. 


Table  18:  Objective  B-l.l  Data  Matrix 


Measure 

Source 

Product 

Objective  B-l.l:  Assess  Usability 

Measure  B-l -1-1:  User  rating  of  the  video  quality  of  the 
identified  technology 

Interview,  Questionnaire 

Table,  Text 

Measure  B-l-1-2:  User  rating  of  the  timeliness  of  data 
received/sent  using  the  identified  technology 

Interview,  Questionnaire 

Table,  Text 

Measure  B-l -1-3:  User  rating  of  the  overall  look  and 
feel  of  the  user  interface  of  the  identified  technology 

Interview,  Questionnaire 

Text,  Bar  Chart 

Objective  B-1.2:  Assess  Reliability 

This  objectives  seeks  to  assess  the  reliability  of  the  identified  technologies.  In  this  context  reliability 
includes  ability  of  the  technology  to  perform  tasks  without  losing  functionality,  ability  of  the 
identified  technology  to  provide  accurate  information,  and  the  user  rating  of  the  overall  reliability  of 
the  identified  technology. 


Table  19:  Objective  B-1.2  Data  Matrix 


Measure 

Source 

Product 

Objective  B-1.2:  Assess  Reliability 

Measure  B-l -1-1:  Number  and  type  of  reliability  issues 
for  the  identified  technology 

Interview,  Questionnaire 
Event  Logs 

Table,  Text 

Measure  B-l-1-2:  User  rating  of  the  accuracy  of  the  data 
provided  by  the  identified  technology 

Interview,  Questionnaire 

Table,  Text 

Measure  B-l -1-3:  User  rating  of  the  overall  reliability  of 
the  identified  technology 

Interview,  Questionnaire 

Text,  Bar  Chart 
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Objective  B-1.3:  Assess  Training 

This  objectives  seeks  to  assess  the  effectiveness  of  the  training  provided  for  each  of  the  identified 
technologies.  Training  will  include  classroom  instruction  as  well  as  hands-on  experience  with 
the  system. 


Table  20:  Objective  B-1.3  Data  Matrix 


Measure 

Source 

Product 

Objective  B-1.3:  Assess  Training 

Measure  B- 1-3-1:  Time  required  to  train  users 

Event  Log 

Table 

Measure  B-l-3-2:  User  rating  of  classroom  training 

Questionnaire,  Interviews 

Bar  Chart,  Text 

Measure  B-l-3-3:  User  rating  of  training  documents 

Questionnaire,  Interviews 

Bar  Chart,  Text 

Functional  Area  B-2:  Mission  Impact 

Do  the  DAVION  Applications  have  a  positive  impact  on  mission  accomplishment? 
Objective  B-2.1:  Assess  Impact  on  Situational  Awareness 

This  objectives  seeks  to  assess  the  impact  of  the  identified  technologies  on  situational  awareness.  In 
this  context  a  positive  impact  on  situational  awareness  is  defined  as  providing  the  user  valuable 
information  that  will  help  accomplish  the  mission  without  negatively  hindering  that  mission. 

Data  collectors  will  conduct  interviews  and/or  surveys  will  be  utilized  to  collect  subjective  data  from 
users. 


Table  21:  Objective  B-2.1  Data  Matrix 


Measure 

Source 

Product 

Objective  B-2.1:  Assess  Impact  on  Situational  Awareness 

Measure  B-l-1-1:  User  rating  of  the  impact  of  the 
identified  technology  on  improving  situational 
awareness 

Interview,  Questionnaire 

Table,  Text,  Bar 
Chart 

Measure  B-l-1-2:  User  rating  of  the  impact  of  the 
overall  usefulness  of  the  identified  technology 

Interview,  Questionnaire 

Table,  Text,  Bar 
Chart 

Objective  B-2.2:  Assess  Impact  on  Decision  Making 

This  objectives  seeks  to  assess  the  impact  of  the  identified  technologies  on  decision  making.  In  this 
context  a  positive  impact  on  decision  making  is  defined  as  providing  the  user  valuable  information 
that  will  help  the  user  make  decisions  important  to  accomplishing  the  mission. 


Table  22:  Objective  B-2.2  Data  Matrix 


Measure 

Source 

Product 

Objective  B-2.2:  Assess  Impact  on  Decision  Making 

Measure  B-2 -2-1 :  User  rating  of  the  impact  of  the 
identified  technology  on  improving  decision  making 

Interview,  Questionnaire 

Table,  Text,  Bar 
Chart 
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Davion  Survey  Results 

This  section  provides  the  DAVION  survey  feedback  collected  from  participating  users  during  the 
OUE  14.2  technology  integration  event. 

Functional  Area  B-l:  Suitability 

Measure  B-l-1-1:  User  rating  of  the  video  quality  of  the  identified  technology 


Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 

Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


The  display  quality  of  the  DAVION  Chat  capability  was  good. 

7 
6 
5 
4 
3 
2 
1 
0 

Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 
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The  display  quality  of  the  D AVION  WhiteBoarding  capability  was  good. 


Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


If  any  technologies  were  not  ranked  as  Largely  Agree  or  Completely  Agree  in  relation  to  Video 
Quality  please  provide  comments  to  clarify  your  ranking: 

•  The  video  quality  could  be  better,  maybe  HD 

•  Photo  quality  was  good 

•  DAVION  chat  was  simple  and  easy  to  use 

•  The  WhiteBoarding  can  be  improved.  Due  to  size  of  user’s  fingers,  and  not  being  able  to 
zoom  in  on  the  map,  at  times  drawing  is  not  accurate. 


Measure  B-l-1-2:  User  rating  of  the  timeliness  of  data  received/sent  using  the  identified 
technology 


The  time  required  to  send  and/or  receive  data  using  the  DAVION  Map  capability  was 

acceptable. 


5 


Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 
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The  time  required  to  send  and/or  receive  data  using  the  D AVION  Augmented  Reality 

capability  was  acceptable. 

6 


Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


The  time  required  to  send  and/or  receive  data  using  the  DAVION  Chat  capability  was 
acceptable. 

6 


Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


The  time  required  to  send  and/or  receive  data  using  the  DAVION  WhiteBoarding  capability 
was  adequate. 

6 

5 

4 

3 

2 

1 

0 

Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


If  any  technologies  were  not  ranked  as  Largely  Agree  or  Completely  Agree  in  relation  to 
Timeliness  please  provide  comments  to  clarify  your  ranking: 

•  At  times  the  reality  capability  detection  would  be  late/lag 

•  The  Chat  capability  was  almost  instantaneous 
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•  WiFi  was  unreliable,  makes  it  hard  to  send  text/pictures 

•  There  was  a  delay  in  video  feed  and  in  aggression  alerts 

•  DAVION  Map  would  take  some  time  to  show  blue  forces  moving 

•  The  accuracy  of  the  location  pin  improved  during  the  event  and  is  a  lot  faster 

•  The  Map  took  a  while  to  refresh  and  keep  me  updated  on  pins  and  other  points  on  the 
map 

Measure  B-l-1-3:  User  rating  of  the  overall  look  and  feel  of  the  user  interface  of  the 
identified  technology 


The  overall  Look  and  Feel  of  the  DAVION  Map  capability  was  good. 


Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


The  overall  Look  and  Feel  of  the  DAVION  Chat  capability  was  good. 


Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 
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The  overall  Look  and  Feel  of  the  DAVION  WhiteBoarding  capability  was  good. 


Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


If  any  technologies  were  not  ranked  as  Largely  Agree  or  Completely  Agree  in  relation  to  Look 
and  Feel  please  provide  comments  to  clarify  your  ranking: 

•  Simple  and  user  friendly 

•  Sometimes  it  didn’t  update,  the  whiteboard  looks  sloppy 

Measure  B-l-1-1:  Number  and  type  of  reliability  issues  for  the  identified  technology 

•  Various  reliability  issues  were  logged  during  the  integration  of  the  technologies.  Issues 
were  also  logged  during  training  and  vignettes.  The  number  and  type  of  each  were  not 
logged  as  the  majority  of  issues  were  tied  to  the  dropping  of  the  Wi-Fi  signal.  The  PGSS 
Mobile  Client  also  crashed  a  couple  of  times  during  the  event.  The  VA  system  also 
encountered  some  issues  due  to  integration  problem  and  a  shorted  timeline  for  training 
the  software.  Additional  details  can  be  found  in  the  “Technology  Integration”  section  of 
the  report. 

Measure  B-l-1-2:  User  rating  of  the  accuracy  of  the  data  provided  by  the  identified 
technology 


The  Accuracy  of  the  DAVION  Map  capability  was  good. 
5 


Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 
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The  Accuracy  of  the  D AVION  Augmented  Reality  capability  data  was  good. 


Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


The  Accuracy  of  the  D AVION  WhiteBoarding  capability  data  was  good. 


Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


If  any  technologies  were  not  ranked  as  Largely  Agree  or  Completely  Agree  in  relation  to 
Accuracy  please  provide  comments  to  clarify  your  ranking: 

•  Trying  to  see  the  points  that  were  meant  to  be  right  on  a  specific  item  or  area  was  off  by  a 
lot 

•  The  DAVION  Map  accuracy  is  quite  slow  updating,  even  when  it  refreshes 

•  The  Augmented  Reality  was  bad,  it  linked  to  another  location 

•  The  DAVION  has  improved,  since  starting  the  event,  based  on  the  convenience  of 
chatting  from  location  to  location  and  it’s  good  to  use  the  Chat  application  to  make  things 
easier  and  clearer  instead  of  using  the  comms  because  at  times  there  might  be  distortion 
of  voice  and  you  can’t  pass  the  message  clearly 

•  The  DAVION  Augmented  Reality  was  not  that  good.  Sometimes  it  would  appear  all 
around  and  not  to  the  place  we  pinned 
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•  The  Map  is  not  accurate,  places  like  friendly  locations  were  off  by  100  meters,  if  it  event 
updated.  Half  of  the  time  the  information  was  not  up  to  date  on  the  friendly  location. 
Augmented  Reality  was  also  not  accurate  to  the  exact  locations  of  the  pins.  Great  if 
looking  for  a  city,  bad  if  looking  for  a  person  or  suspected  IED.  Chat  worked  as  long  as 
WiFi  worked 

•  Augmented  Reality  was  not  accurate 

•  At  times  the  accuracy  of  the  Map  would  lag  when  blue  forces  were  moving 

Measure  B-l-1-3:  User  rating  of  the  overall  reliability  of  the  identified  technology 


The  overall  Reliability  (functioned  as  expected  without  crashing )  of  the  D AVION  Map 

capability  was  good. 
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Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


The  overall  Reliability  of  the  D AVION  Augmented  Reality  capability  was  good. 


8 

7 
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5 

4 

3 
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1 
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Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


The  overall  Reliability  of  the  D AVION  Chat  capability  was  good. 


Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 
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The  overall  Reliability  of  the  D AVION  WhiteBoarding  capability  was  good. 


8 

7 

6 

5 

4 

3 

2 

i  4 - 

0 

Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


If  any  technologies  were  not  ranked  as  Largely  Agree  or  Completely  Agree  in  relation  to 
Accuracy  please  provide  comments  to  clarify  your  ranking: 

•  WiFi  crashed  often,  which  effected  Chat  and  Maps 

•  Delayed  updates  for  aggression  detection 

Measure  B-l-3-1:  Time  required  to  train  users 

•  Verbal  training  session  was  conducted  in  less  than  1  hour.  Often  the  users  picked  up  on 
the  applications  within  a  few  minutes  and  used  the  remaining  time  to  ask  questions  and 
explore  the  technology. 

Measure  B- 1-3-2:  User  rating  of  classroom  and  hands-on  training 

The  classroom  and  hands-on  training  provided  for  the  D AVION  applications  was  good. 


Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


How  would  you  suggest  training  be  improved  for  any  or  all  of  the  applications  you  used 

during  OUE  14.2? 

•  If  would  be  great  if  the  user  could  confirm  that  an  Chat  message  had  been  received,  like 
with  WhatsApp 
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•  Use  more  real  world  situations  to  test  the  gear  and  how  well  it  will  function 

•  More  realistic  training  scenarios,  better  WiFi  capabilities 

•  Training  and  the  teaching  of  using  the  apps  was  good 

•  More  realistic  scenarios,  and  try  to  have  everything  up  and  working  to  help  the  training 
feel  more  real 

•  The  scenarios,  though  realistic  in  nature,  are  not  applied  in  a  realistic  way 

Measure  B- 1-3-4:  User  rating  of  training  documents 
The  training  documents/presentations  provided  for  the  D AVION  applications  were  helpful. 


7 


Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 

Functional  Area  B-2:  Mission  Impact 

Measure  B-l-1-1:  User  rating  of  the  impact  of  the  identified  technology  on  improving 
situational  awareness 

The  D AVION  Map  capability  had/would  have  a  positive  impact  on  situational  awareness. 


Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 
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The  D AVION  Augmented  Reality  capability  had/would  have  a  positive  impact  on  situational 

awareness. 


Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


The  D AVION  Chat  capability  had/would  have  a  positive  impact  on  situational  awareness. 


Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


The  D AVION  WhiteBoarding  capability  had/would  have  a  positive  impact  on  situational 

awareness. 


Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


If  any  technologies  were  not  ranked  as  Largely  Agree  or  Completely  Agree  in  relation  to 
Situational  Awareness  please  provide  comments  to  clarify  your  ranking. 

•  Information  sent  though  Chat,  i.e.  pictures,  was  good 

•  The  user  will  have  a  better  understanding  of  what  is  happening  during  operations 
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Measure  B-l-1-2:  User  rating  of  the  impact  of  the  overall  usefulness  of  the  identified 
technology  in  urban  environments 

The  D AVION  Map  capability  would  be  useful  in  urban  environments 


Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


The  D AVION  Augmented  Reality  capability  would  be  useful  in  urban  environments 
5 


Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


The  D AVION  Chat  capability  would  be  useful  in  urban  environments 

6 


Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 
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7 


The  DAVION  WhiteBoarding  capability  would  be  useful  in  urban  environments 


Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


If  any  technologies  were  not  ranked  as  Largely  Agree  or  Completely  Agree  in  relation  to 
operating  in  urban  environments  please  provide  comments  to  clarify  your  ranking. 

•  It  would  be  simpler  to  radio  in  unless  we  needed  to  remain  clandestine  then  it  could  be 
better  choice 

Measure  B-2-2-1:  User  rating  of  the  impact  of  the  identified  technology  on  improving 
decision  making 

The  DAVION  Map  capability  would  provide  valuable  information  to  help  the  user  make 
decisions  important  to  accomplishing  missions. 

8 


Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 
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The  D AVION  Augmented  Reality  capability  would  provide  valuable  information  to  help  the 
user  make  decisions  important  to  accomplishing  missions. 


Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


The  D AVION  Chat  capability  would  provide  valuable  information  to  help  the  user  make 
decisions  important  to  accomplishing  missions. 


Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


The  D AVION  WhiteBoarding  capability  would  provide  valuable  information  to  help  the  user 
make  decisions  important  to  accomplishing  missions. 

6 


Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 
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If  any  technologies  were  not  ranked  as  Largely  Agree  or  Completely  Agree  in  relation  to 
Decision  Making  please  provide  comments  to  clarify  your  ranking. 

•  The  blue  force  tracking  would  help  the  commander  monitor  the  position  of  his  troops,  but 
time  accuracy  must  be  improved 

•  Should  have  a  stylus  for  whiteboarding  for  better  accuracy 

•  When  it  comes  to  passing  information  we  already  have  radio  communications  we  can 
use.  If  we  needed  to  remain  clandestine  chat  wouldn’t  be  a  better  choice,  however,  if  on 
a  patrol  it  would  be  better  just  to  radio  in  versus  trying  to  send  text 

Additional  Survey  Questions 

Would  the  D AVION  applications  be  a  valuable  tool  in  HADR  environments?  Explain. 

•  Position  of  troops 

•  Quick  infonnation  on  sighted  riots  or  “change  of  baseline” 

•  Yes,  it  would  help  organize  and  give  better  situational  awareness  to  anyone  using  it  and 
could  help  make  better  decisions 

•  Yes,  the  ability  to  see  mission  tasks  of  multiple  squads,  and  gain  better  situational 
awareness  will  allow  commanders  to  make  real  time  decisions. 

•  Pax  detection  and  aggression  alerts  provide  more  safety,  SA,  and  security  for  marines 
especially  in  a  MOUT  environment 

•  DAVION  applications  would  be  valuable  when  we  can’t  do  our  radio  chat  in  some  places 
or  we  need  to  be  quiet  in  some  places,  we  would  markout  the  place  and  use  chat 

•  Yes,  it  would  be  useful  as  a  just  in  case  if  comms  or  radio  were  down.  We  could  use  the 
Chat  application  to  communicate  our  whereabouts  to  another  person  at  another  location 
and  pin  our  location  and  markout  where  we  are  at 

•  Yes  it  will,  because  if  needed  to  more  people  or  personnel  from  one  spot  to  another  both 
Map  and  Whiteboarding  would  work  for  that 

•  Yes,  we  could  drop  pins  in  areas  in  need  of  certain  supplies  and  also  use  the  pins  to 
categorize  certain  areas,  i.e.  critical,  urgent,  stable,  etc. 

Are  there  any  other  environments  where  any  or  all  of  the  PGSS  Mobile  COP  applications 

would  be  valuable? 

•  In  any  landscape  environment  where  there  could  be  possible  IEDs  the  AR  funcation 
would  be  very  helpful  in  alerting  other  united  of  possible  IED  location 

•  Yes,  any  environment  would  be  great  to  use  the  DAVION  technology 

•  It  would  be  good  during  conventional  ops  as  it  would  be  wasier  to  communicate  instead 
of  using  the  comms  at  all  times 

•  Detecting  enemy  and  blue  forces  in  the  area 

•  It’s  good  for  conventional  so  that  we  can  mark  out  the  enemy  harboring  area  and  using 
the  Whiteboarding  app  to  send  it  to  everybody  so  we  know  how  we  are  going  to  attack 
the  enemy 

•  MOUT  environments  would  benefit  most  from  DAVION  applications  due  to  the 
requirement  for  WiFi  connection.  The  DAVION  would  not  be  of  great  use  in  other 
environments 

•  Yes,  it  could  be  useful  in  any  environment 


OUE  14.2  Final  Report 


74 


•  Conventional  (jungle),  Live  Firing  (company/platoon) 

Are  there  any  changes  you  would  make  to  any  of  the  D AVION  applications? 

•  Blue  tracks  like  those  they  have  in  WhatsApp 

•  A  more  powerful  WiFi  network 

•  Major  upgrade  to  the  WiFi  and  make  the  3  apps  interconnected  to  allow  easier  and 
quicker  access  and  passing  on  information 

•  Improve  the  Whiteboarding  app,  if  others  are  making  changes  to  the  Whiteboard  send  an 
alert  so  the  users  knows 

•  Better  range  on  the  Chat,  because  sometimes  it  will  not  receive  or  send  out  if  the  distance 
is  pretty  far  apart.  And  it  does  automatic  logout  when  not  in  use  after  a  while  which 
causes  miscommunications.  This  will  delay  the  mission. 

•  Try  to  combine  Map  and  Whiteboarding  to  one  app,  would  make  it  faster  to  use  and  less 
of  a  hassle 

•  Better  WiFi  connection 

•  Make  the  3  apps  into  a  single  app 

Are  there  any  additional  applications  that  would  add  value  to  the  D AVION  applications 

package? 

•  Hands  free  communication,  Bluetooth 

•  Adding  a  voice  recorder  button  just  in  case  if  there  is  an  IED  spotted  and  the  person 
might  be  too  nervous  to  react  and  type  they  can  press  the  record  button  and  report  what 
they  have  seen,  it  would  be  much  easier 

•  Maybe  a  reinforcement  button  for  when  there  is  a  need  for  urgency,  so  we  don’t  need  to 
type 

•  Bluetooth  headset  capability,  some  sort  of  hands  free  communication 

•  Bluetooth  headset  cability 

•  Aerial  surveillance  to  over  view  the  objective/area  of  ops 
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PGSS  MOBILE  CLIENT  ASSESSMENT  RESULTS 


This  section  provides  details  on  the  functional  areas,  objectives,  measures  used  to  collect  data  on 
the  PGSS  Mobile  Client.  It  also  includes  the  survey  feedback  collected  during  the  event. 

PGSS  Mobile  Client  Functional  Areas,  Objectives,  and  Measures 

This  section  identifies  the  measures  used  to  collect  data  on  the  suitability  of  the  PGSS  Mobile  Client 
during  OUE  14.2.  The  TEC  assessment  team  addressed  the  measures  using  the  data  sources 
identified  in  the  following  tables.  All  results  are  supported  by  observations  and  user  comments  when 
available. 

Functional  Area  B-l:  Suitability 

Are  the  PGSS  Mobile  Client  suitable  for  supporting  operations  in  urban  environments? 
Objective  B-l.l:  Assess  Usability 

This  objectives  seeks  to  assess  the  usability  of  the  identified  technologies.  In  this  context  usability  is 
includes  the  overall  look  and  feel,  video  quality,  and  timeliness  of  data. 


Table  23:  Objective  B-l.l  Data  Matrix 


Measure 

Source 

Product 

Objective  B-l.l:  Assess  Usability 

Measure  B-l -1-1:  User  rating  of  the  video  quality  of  the 
identified  technology 

Interview,  Questionnaire 

Table,  Text 

Measure  B-l-1-2:  User  rating  of  the  timeliness  of  data 
received/sent  using  the  identified  technology 

Interview,  Questionnaire 

Table,  Text 

Measure  B-l -1-3:  User  rating  of  the  overall  look  and 
feel  of  the  user  interface  of  the  identified  technology 

Interview,  Questionnaire 

Text,  Bar  Chart 

Objective  B-1.2:  Assess  Reliability 

This  objectives  seeks  to  assess  the  reliability  of  the  identified  technologies.  In  this  context  reliability 
includes  ability  of  the  technology  to  perform  tasks  without  losing  functionality,  ability  of  the 
identified  technology  to  provide  accurate  information,  and  the  user  rating  of  the  overall  reliability  of 
the  identified  technology. 


Table  24:  Objective  B-1.2  Data  Matrix 


Measure 

Source 

Product 

Objective  B-1.2:  Assess  Reliability 

Measure  B-l -1-1:  Number  and  type  of  reliability  issues 
for  the  identified  technology 

Interview,  Questionnaire 
Event  Logs 

Table,  Text 

Measure  B-l-1-2:  User  rating  of  the  accuracy  of  the  data 
provided  by  the  identified  technology 

Interview,  Questionnaire 

Table,  Text 

Measure  B-l-1-3:  User  rating  of  the  overall  reliability  of 
the  identified  technology 

Interview,  Questionnaire 

Text,  Bar  Chart 
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Objective  B-1.3:  Assess  Training 

This  objectives  seeks  to  assess  the  effectiveness  of  the  training  provided  for  each  of  the  identified 
technologies.  Training  will  include  classroom  instruction  as  well  as  hands-on  experience  with 
the  system. 


Table  25:  Objective  B-1.3  Data  Matrix 


Measure 

Source 

Product 

Objective  B-1.3:  Assess  Training 

Measure  B- 1-3-1:  Time  required  to  train  users 

Event  Log 

Table 

Measure  B-l-3-2:  User  rating  of  classroom  training 

Questionnaire,  Interviews 

Bar  Chart,  Text 

Measure  B-l-3-3:  User  rating  of  training  documents 

Questionnaire,  Interviews 

Bar  Chart,  Text 

Functional  Area  B-2:  Mission  Impact 

Do  the  DAVION  Applications  have  a  positive  impact  on  mission  accomplishment? 
Objective  B-2.1:  Assess  Impact  on  Situational  Awareness 

This  objectives  seeks  to  assess  the  impact  of  the  identified  technologies  on  situational  awareness.  In 
this  context  a  positive  impact  on  situational  awareness  is  defined  as  providing  the  user  valuable 
information  that  will  help  accomplish  the  mission  without  negatively  hindering  that  mission. 

Data  collectors  will  conduct  interviews  and/or  surveys  will  be  utilized  to  collect  subjective  data  from 
users. 


Table  26:  Objective  B-2.1  Data  Matrix 


Measure 

Source 

Product 

Objective  B-2.1:  Assess  Impact  on  Situational  Awareness 

Measure  B-l-1-1:  User  rating  of  the  impact  of  the 
identified  technology  on  improving  situational 
awareness 

Interview,  Questionnaire 

Table,  Text,  Bar 
Chart 

Measure  B-l-1-2:  User  rating  of  the  impact  of  the 
overall  usefulness  of  the  identified  technology 

Interview,  Questionnaire 

Table,  Text,  Bar 
Chart 

Objective  B-2.2:  Assess  Impact  on  Decision  Making 

This  objectives  seeks  to  assess  the  impact  of  the  identified  technologies  on  decision  making.  In  this 
context  a  positive  impact  on  decision  making  is  defined  as  providing  the  user  valuable  information 
that  will  help  the  user  make  decisions  important  to  accomplishing  the  mission. 


Table  27:  Objective  B-2.2  Data  Matrix 


Measure 

Source 

Product 

Objective  B-2.2:  Assess  Impact  on  Decision  Making 

Measure  B-2 -2-1 :  User  rating  of  the  impact  of  the 
identified  technology  on  improving  decision  making 

Interview,  Questionnaire 

Table,  Text,  Bar 
Chart 
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PGSS  Mobile  Client  Survey  Results 

This  section  provides  the  PGSS  Mobile  Client  survey  feedback  collected  from  participating  users 

during  the  OUE  14.2  technology  integration  event. 

Functional  Area  B-l:  Suitability 

Measure  B-l-1-1:  User  rating  of  the  video  quality  of  the  identified  technology 

The  Video  quality  of  the  PGSS  Mobile  Client  Streaming  Video  capability  was  good. 

7 
6 
5 
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Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


The  Video  quality  of  the  PGSS  Mobile  Client  Chat  capability  was  good. 


Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


The  Video  quality  of  the  PGSS  Mobile  Client  Maps  capability  was  good. 
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Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 
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The  quality  of  photos  taken,  sent,  and  received  by  the  PGSS  Mobile  Client  was  good. 


Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


The  quality  of  the  telephone  calls  made  within  the  PGSS  Mobile  Client  user  network  was 

good. 


Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


The  quality  of  the  telephone  calls  made  outside  of  the  PGSS  Mobile  Client  user  network  was 

good. 


Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 
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If  any  technologies  were  not  ranked  as  Largely  Agree  or  Completely  Agree  in  relation  to  Video 
Quality  please  provide  comments  to  clarify  your  ranking: 

•  The  video  quality  of  the  map  can  be  improved  by  allowing  the  phones  to  zoom  in  more, 
thus  planning  the  pin  would  be  more  accurate  and  allow  users  to  clearly  see  the  map 
because  every  small  detail  in  urban  operations  is  important 

•  Maps  seems  like  they  were  pulled  from  Google  Maps,  not  updated  and  get  blurry  at 
points 

•  Pictures  taken  were  taking  a  while  to  show  up  on  the  library  so  other  users  could  not  see 
it  at  the  exact  moment  when  it  was  taken 


Measure  B-l-1-2:  User  rating  of  the  timeliness  of  data  received/sent  using  the  identified 
technology 

The  time  required  to  send  and/or  receive  data  using  the  PGSS  Mobile  Client  GPS  Tracking 

capability  was  adequate. 


Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


The  time  required  to  send  and/or  receive  data  using  the  PGSS  Mobile  Client  Streaming  Video 

capability  was  adequate. 
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Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 
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The  time  required  to  send  and/or  receive  data  using  the  PGSS  Mobile  Client  Chat  capability 

was  adequate. 

5 


Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


The  time  required  to  send  and/or  receive  data  using  the  PGSS  Mobile  Client  Maps  capability 

was  adequate. 

6 


Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


If  any  technologies  were  not  ranked  as  Largely  Agree  or  Completely  Agree  in  relation  to 
Timeliness  please  provide  comments  to  clarify  your  ranking: 

•  When  trying  to  setup  POI  tags  or  waypoints  the  system  failed  at  some  points 

•  Could  have  a  program  that  tells  you  whether  the  other  party  has  received  your  message  so 
that  the  sender  is  aware,  i.e.  WhatsApp 


OUE  14.2  Final  Report 


81 


Measure  B-l-1-3:  User  rating  of  the  overall  look  and  feel  of  the  user  interface  of  the 
identified  technology 

The  overall  Look  and  Feel  of  the  PGSS  Mobile  Client  Streaming  Video  capability  was  good. 


Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


The  overall  Look  and  Feel  of  the  PGSS  Mobile  Client  Chat  capability  was  good. 


Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


The  overall  Look  and  Feel  of  the  PGSS  Mobile  Client  Maps  capability  was  good. 


Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 
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The  overall  Look  and  Feel  of  the  PGSS  Mobile  Client  Main  Menu  Interface  was  good. 
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Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


If  any  technologies  were  not  ranked  as  Largely  Agree  or  Completely  Agree  in  relation  to  Look 
and  Feel  please  provide  comments  to  clarify  your  ranking: 

•  The  mobile  chat  could  be  improved  by  popping  a  small  window  similar  to  WhatsApp 
when  you  are  in  “video”  or  “map”  mode.  That  way  we  know  immediately  there  is  a 
message  received  and  who  sent  the  message.  This  would  save  time  because  everything 
about  urban  operations  happens  in  a  split  second.  So  a  few  seconds  could  make  a  lot  of 
difference. 

•  Weak  WiFi  sometimes 

•  There  is  no  need  for  the  system  to  ask  whether  to  “Chat”  once  the  user  has  pressed  for  the 
other  party 

•  The  Startup/loading  of  the  map  could  be  faster 

•  The  map  should  be  integrated  with  the  client  streaming  video  as  both  have  the  same 
purpose 

•  Should  name  it  surveillance/map  streaming  or  something 

Measure  B-l-1-1:  Number  and  type  of  reliability  issues  for  the  identified  technology 

•  Various  reliability  issues  were  logged  during  the  integration  of  the  technologies.  Issues 
were  also  logged  during  training  and  vignettes.  The  number  and  type  of  each  were  not 
logged  as  the  majority  of  issues  were  tied  to  the  dropping  of  the  Wi-Fi  signal,  and  time 
with  users  was  limited.  The  PGSS  Mobile  Client  also  crashed  a  couple  of  times  during 
the  event.  The  VA  system  also  encountered  some  issues  due  to  integration  problem  and  a 
shorted  timeline  for  training  the  software.  Additional  details  can  be  found  in  the 
“Technology  Integration”  section  of  the  report. 
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Measure  B-l-1-2:  User  rating  of  the  accuracy  of  the  data  provided  by  the  identified 
technology 

The  Accuracy  of  the  PGSS  Mobile  Client  Map  Blue  Force  Tracking  capability  data  was  good. 


Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


The  Accuracy  of  the  PGSS  Mobile  Client  Streaming  Video  capability  data  was  good. 


Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


The  Accuracy  of  the  PGSS  Mobile  Client  Chat  capability  data  was  good. 
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Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 
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The  Accuracy  overall  PGSS  Mobile  Client  Maps  capability  was  data  good. 


Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


If  any  technologies  were  not  ranked  as  Largely  Agree  or  Completely  Agree  in  relation  to 
Accuracy  please  provide  comments  to  clarify  your  ranking: 

•  The  blue  force  tracking  was  good  but  sometimes  needed  some  time  to  update 

•  Due  to  the  limitation  of  zoom  it’s  very  hard  to  get  an  accurate  pinpoint  location 

•  It  will  be  better  if  the  streaming  videos  were  in  real  time 

•  It  would  be  great  if  the  map  could  have  options  of  satellite  or  map  view  like  those  used  in 
Google  Maps 

•  The  user  should  be  made  aware  if  their  message  was  received  by  the  other  party 

•  Maps  could  be  improved  by  allowing  more  zoom  to  the  exact  detail  on  the  ground  to  be 
accurate  for  pinpoint,  description  and  photos  at  that  pinpoint 

Measure  B-l-1-3:  User  rating  of  the  overall  reliability  of  the  identified  technology 

The  overall  Reliability  ( information  displayed  as  expected  and  did  not  crash )  of  the  PGSS 
Mobile  Client  Blue  Force  Tracking  capability  was  good. 


Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 
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The  overall  Reliability  of  the  PGSS  Mobile  Client  Streaming  Video  capability  was  good. 


Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


The  overall  Reliability  of  the  PGSS  Mobile  Client  Chat  capability  was  good. 


Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


The  overall  Reliability  of  the  PGSS  Mobile  Client  Maps  capability  was  good. 


Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 
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The  overall  Reliability  of  the  PGSS  Mobile  Client  telephone  capability  was  good. 
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Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


If  any  technologies  were  not  ranked  as  Largely  Agree  or  Completely  Agree  in  relation  to 
Accuracy  please  provide  comments  to  clarify  your  ranking: 

•  Update  the  phones  with  more  options,  and  compatibility  with  HDMI 

•  The  program  crashed  a  few  times 

•  The  coverage  of  the  program  at  a  far  distance  was  not  accurate.  WiFi  ability  is  not  good 
when  distance  is  more  than  150m,  thus  not  allowing  for  information  exchange 

•  The  telephone  capability  was  not  functioning  during  training 

•  The  system  crashed  a  few  times  during  operations 

•  The  client  maps  are  a  bit  useless  with  the  maps  show  in  the  client  streaming  video,  should 
integrate  it 

•  Video  needed  some  minutes  to  update  and  connect  to  main  or  you  would  have  to  reset  the 
app  in  order  to  see  the  video 

•  Maps  can  get  easier  to  use  according  to  the  area  you  are  operating.  I  didn’t  like  that  any 
time  I  got  out  and  back  in  the  map  section  it  would  take  me  all  the  way  zoomed  out  and 
takes  seconds  to  zoom  back  in  on  the  objective 

Measure  B- 1-3-1:  Time  required  to  train  users 

•  Verbal  training  session  was  conducted  in  less  than  1  hour.  Often  the  users  picked  up  on 
the  applications  within  a  few  minutes  and  used  the  remaining  time  to  ask  questions  and 
explore  the  technology. 
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Measure  B- 1-3-2:  User  rating  of  classroom  and  hands-on  training 
Measure  B- 1-3-4:  User  rating  of  training  documents 

The  classroom  and  hands-on  training  provided  for  the  PGSS  Mobile  Client  was  good. 
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Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


The  training  documents/presentations  provided  for  the  PGSS  Mobile  Client  applications  were 

helpful. 
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Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


How  would  you  suggest  training  be  improved  for  any  or  all  of  the  applications  you  used 

during  OUE  14.2? 

•  I  believe  it’s  simple  enough,  awesome  to  use,  and  field  test,  easy  to  learn  the  tech  as  is.  I 
jumped  in  late,  and  I  was  able  to  pick  it  up  in  30  min 

•  Faster  streaming,  clearer  picture,  needs  better  WiFi  reliability 

•  Notifications 

•  Wrist  pouch 

•  Better  WiFi 

•  Better  screen  protectors  as  we  can’t  see  when  the  sun  is  too  bright 

•  Pop  up  of  photo  indicated  at  where  we  mark  our  waypoint 

•  Better  zooming  on  blue  force  map  so  we  could  markout  waypoints  more  accurately 

•  I  thought  the  training  was  good.  The  system  itself  is  pretty  straight  forward  and  easy  to 
understand  so  the  improvements  don’t  need  to  made  there 

•  Maybe  run  more  scenarios  but  everything  was  good 
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Functional  Area  B-2:  Mission  Impact 

Measure  B-l-1-1:  User  rating  of  the  impact  of  the  identified  technology  on  improving 
situational  awareness 

The  PGSS  Mobile  Client  Streaming  Video  capability  had/would  have  a  positive  impact  on 

situational  awareness. 
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Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


The  PGSS  Mobile  Client  Chat  capability  had/would  have  a  positive  impact  on  situational 

awareness. 


Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


The  PGSS  Mobile  Client  Maps  capability  ( including  Blue  Force  Tracking )  had/would  have 

positive  impact  on  situational  awareness. 
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Completely  Largely 
Disagree  Disagree 


Somewhat  Somewhat  Largely 
Disagree  Agree  Agree 


Completely  Not 
Agree  Applicable 
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The  PGSS  Mobile  Client  telephone  capability  had/would  have  a  positive  impact  on  situational 

awareness. 
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Completely  Largely 
Disagree  Disagree 


Somewhat  Somewhat  Largely 
Disagree  Agree  Agree 


Completely  Not 
Agree  Applicable 


If  any  technologies  were  not  ranked  as  Largely  Agree  or  Completely  Agree  in  relation  to 
Situational  Awareness  please  provide  comments  to  clarify  your  ranking. 

•  Must  have  notification  (vibration  or  ping) ! 

•  During  missions  the  soldier  on  the  ground  must  have  situational  awareness  and  not  look 
at  the  phone  screen  all  of  the  time 

•  The  telephone  capability  seems  really  helpful  for  faster  information  exchange 

Measure  B-l-1-2:  User  rating  of  the  impact  of  the  overall  usefulness  of  the  identified 
technology  in  urban  environments 

The  PGSS  Mobile  Client  Streaming  Video  capability  would  be  useful  in  urban  environments 


Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 
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The  PGSS  Mobile  Client  Chat  capability  would  be  useful  in  urban  environments 


Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


The  PGSS  Mobile  Client  Maps  capability  would  be  useful  in  urban  environments 
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Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


The  PGSS  Mobile  Client  telephone  capability  would  be  useful  in  urban  environments 
5 


Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


If  any  technologies  were  not  ranked  as  Largely  Agree  or  Completely  Agree  in  relation  to 
operating  in  urban  environments  please  provide  comments  to  clarify  your  ranking. 

•  It  would  be  useful  if  it  is  stealth  and  only  on  deliberate  attack.  It  wouldn’t  be  useful  if  it 
was  a  hasty  attack 
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Measure  B-2-2-1:  User  rating  of  the  impact  of  the  identified  technology  on  improving 
decision  making 

The  PGSS  Mobile  Client  Map  Blue  Force  Tracking  capability  would  provide  valuable  information 
to  help  the  user  make  decisions  important  to  accomplishing  missions. 
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Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 

The  PGSS  Mobile  Client  Streaming  Video  capability  would  provide  valuable  information  to  help 
the  user  make  decisions  important  to  accomplishing  missions. 
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Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


The  PGSS  Mobile  Client  Chat  capability  would  provide  valuable  information  to  help  the  user 
make  decisions  important  to  accomplishing  missions. 


Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 
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The  PGSS  Mobile  Client  Maps  capability  would  provide  valuable  information  to  help  the  user 
make  decisions  important  to  accomplishing  missions. 
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Completely  Largely  Somewhat  Somewhat 
Disagree  Disagree  Disagree  Agree 


Largely  Completely  Not 
Agree  Agree  Applicable 


The  PGSS  Mobile  Client  telephone  capability  would  provide  valuable  information  to  help  the  user 
make  decisions  important  to  accomplishing  missions. 

4 

3 

2 

1 

0 

Completely  Largely  Somewhat  Somewhat  Largely  Completely  Not 
Disagree  Disagree  Disagree  Agree  Agree  Agree  Applicable 


If  any  technologies  were  not  ranked  as  Largely  Agree  or  Completely  Agree  in  relation  to 
Decision  Making  please  provide  comments  to  clarify  your  ranking. 

•  The  Chat  function  will  pop  up  a  window  and  ask  you  to  press  “Chat”  before  you  can  start 
typing  a  message.  I  think  this  is  a  double  step  which  is  kind  of  useless  and  time  wasting 

•  I  have  it  all  at  largely  agree  because  the  technology  did  fail  us  at  the  worst  time,  and 
possibly  at  the  aerostat  is  a  target  for  being  shot  down.  WiFi  connection  wasn’t  that  great 
even  on  peace  time  training. 

•  Chat  capability  is  not  too  valuable  because  it  is  delayed  and  it  takes  longer  to  text  than 
call 

Additional  Survey  Question 

Would  the  PGSS  Mobile  Client  be  a  valuable  tool  in  HADR  environments?  Explain. 

•  Yes,  to  see  a  bigger  area  with  live  streaming  to  see  what  going  on,  and  also  to  pinpoint 
specific  personnel/items 

•  Yes  definitely;  It  allows  blue  force  tracking,  The  pinpoints  on  the  map  would  be  very 
useful  to  show  all  important  locations  be  it  blue  or  red  force,  it  allows  fast  information 
exchange  through  troops  on  the  ground  and  higher  HQ,  and  allows  higher  HQ  to  have  a 
clear  view  of  what  is  happening  on  the  ground 

•  Yes  it  will,  as  HADR  is  more  central  environment,  we  could  use  it  to  scan  the  area  to 
check  for  suspicious  suicide  bombers 
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•  Yes,  you  could  see  how  many  people  may  ned  to  be  rescued  or  how  flooded  an  area  is 
before  you  go  there  or  just  what  the  overall  situation  is  like 

•  Yes,  we  would  have  an  aerial  view  of  operations.  Assistance  could  be  given  a  priority  to 
those  in  areas  needing  it  most 

•  We  could  also  be  updated  if  there  were  to  be  any  riots  that  have  been  formed  up  during 
the  HADR  mission,  e.g.  when  NGOs  are  handing  out  food  rations/medical 

•  Yes,  it  would  be,  you  can  deploy  the  PGSS  and  get  a  live  video  stream  of  what  the 
situation  looks  like,  search  for  survivors,  and  be  able  to  deploy  units  and  maintain  visual 
and  chat  capabilities  with  them  at  all  times. 

Are  there  any  other  environments  where  any  or  all  of  the  PGSS  Mobile  Client  applications 

would  be  valuable? 

•  Combat,  HA,  Border  security,  police  department 

•  Conventional 

•  Desert,  grasslands 

•  Company  operations  base,  platoon  operations  base,  VIP  escorts 

•  Conventional  operations,  the  blue  force  tracking  would  be  good  because  it  is  always  hard 
to  locate  your  blue  forces 

•  Conventional  operations,  the  chat  would  be  good  for  information  exchange  while  still 
adhering  to  field  noise  discipline 

•  Jungle,  desert,  mountains 

•  At  sea,  maybe  having  that  tech  attached  to  scout  helos  and  over  watch  helos  to  give  a 
team  more  of  a  head  count  on  enemy  combatants  before  conducting  a  UBSS  mission 

Are  there  any  changes  you  would  make  to  any  of  the  PGSS  Mobile  applications? 

•  Better  WiFi  and  faster,  clearer  streaming 

•  See  comments  in  previous  questions 

•  The  interface 

Are  there  any  additional  applications  that  would  add  value  to  the  PGSS  Mobile  applications 

package? 

•  Different  types  of  users  in  order  to  classify  different  phones  in  different  categories  so  the 
squad  leader  can  have  more  control,  able  to  zoom  in/out,  text  to  main,  where  other  users 
only  have  the  ability  to  watch  and  text,  making  it  easier  to  have  one  guy  control 

•  Marking  of  red  force  danger  areas 

•  Emergency  buttons 

•  Reset  buttons  for  all  classified  info  in  phones 

•  Larger  coverage  area 

•  In  Chat  if  you  click  a  user  it  transitions  to  chat  instead  of  having  to  click  the  chat  button 

•  Voice  clip 

•  Video  capture 

•  Better  phone  so  you  wouldn’t  have  to  get  so  close  to  potential  threats 

•  Video  calls 

•  Emergency  buttons  like  total  erase,  contacted,  need  reinforcements,  etc. 

•  See  comments  in  previous  questions 
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Summary 


The  purpose  of  the  OUECF  is  to  explore  and  demonstrate  technologies  and  capabilities  for  urban 
operations  in  the  context  of  operationally  relevant  scenarios.  As  part  of  the  that  framework,  OUE 
14.2  was  conducted  in  Piedra  del  Lumbre  (PDL)  Combat  Town,  Camp  Pendleton,  CA  from  5-19 
December  2014  in  conjunction  with  Exercise  Valiant  Mark  14.2.  The  event  consisted  of  an 
integrated  demonstration  of  the  Persistent  Ground  Surveillance  System  (PGSS),  4G  LTE 
communications,  video  analytics,  and  the  DAVION  and  PGSS  Mobile  smart  phone  applications. 
Technologist  from  various  Singaporean  and  U.S.  organizations,  and  users  from  the  Singaporean 
Guard  and  Alpha  Company,  1st  Battalion,  5th  Marine  Regiment  participated  in  the  event. 

The  participating  technologists,  Singaporean  Guard  users,  and  U.S.  Marine  users  all  provided 
feedback  on  the  technologies  that  were  inserted  in  OUE  14.2.  Overall,  the  technologist  viewed  the 
integration  of  the  technologies  as  a  valuable  and  engaging,  collaborative  experience.  Survey  data 
reflected  a  very  positive  outcome,  with  most  users  viewing  all  aspects  of  the  integration  process  as  a 
success.  The  technologist  provided  valuable  feedback  on  how  to  improve  future  integration  events, 
with  the  emphasis  being  on  additional  integration  time,  and  the  participation  of  key  technologist  for 
the  duration  of  the  event.  The  Singaporean  Guards  and  U.S.  Marines  provided  the  technologists  with 
valued  feedback  on  the  functionality,  possible  improvements,  and  streamlining  of  the  capabilities  of 
the  DAVION  applications  and  PGSS  Mobile  Client.  The  survey  data  reflected  that  in  almost  all 
cases  the  users  were  very  happy  with  the  technologies  and  felt  each  would  be  a  valuable  asset  for 
decision  making  and  situational  awareness  in  almost  any  environment. 

A  combined  team  of  Singaporean  Guards  and  U.S.  Marines,  using  the  DAVION  and  PGSS  handsets, 
successfully  conducted  a  live  demonstration  of  some  of  the  features  provided  by  the  applications, 
while  utilizing  the  PGSS  network  and  Video  Analytics  software  on  December  19,  as  part  of  the  DV 
Day.  Visitors  were  also  provided  technology  briefs  and  encouraged  to  ask  questions  and  provide 
feedback  on  the  application  of  each  technology.  The  event  was  attended  by  over  60  guests  including; 
Major  General  Lawrence  D.  Nicholson,  Commanding  General,  1st  Marine  Division,  COL  Lim  Siong 
Tiong,  Head  Concept  Generation  Group,  Future  Systems  and  Technology  Directorate,  COL  Tan 
Cheng  Kwee,  Commander  7  Singapore  Infantry  Brigade  (7  SIB),  Sergeant  Major  David  L.  Jobe, 
Sergeant  Major,  1st  Marine  Division,  and  other  military  and  civilian  visitors  of  note. 

Overall,  OUE  14.2  was  a  successful  event  for  technology  insertion,  integration,  data  collection,  and 
partner  nation  S&T  collaboration  efforts.  The  data  collected  from  this  event  will  help  shape 
continued  technology  development  for  our  warfighters  and  future  PACOM  S&T  engagement  effort. 
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APPENDIX  A:  OUE  14.2  DOCUMENTATION 


OUE  14.2  documents  including;  surveys,  demographics  forms,  event  logs,  and  brochures 
can  be  provided  upon  request.  Please  note  that  the  surveys,  demographics,  and  event  log 
forms  are  blank  forms  to  protect  the  anonymity  of  the  users  and  participants. 


